Re: [HACKERS] Challenges preventing us moving to 64 bit transaction id (XID)?

2017-06-05 Thread Ashutosh Bapat
On Tue, Jun 6, 2017 at 9:48 AM, Craig Ringer wrote: > On 6 June 2017 at 12:13, Ashutosh Bapat > wrote: > >> What happens when the epoch is so low that the rest of the XID does >> not fit in 32bits of tuple header? Or such a case should never arise? > > Storing an ep

Re: [HACKERS] Challenges preventing us moving to 64 bit transaction id (XID)?

2017-06-05 Thread Ashutosh Bapat
On Tue, Jun 6, 2017 at 10:00 AM, Tom Lane wrote: > Ashutosh Bapat writes: >> On Tue, Jun 6, 2017 at 9:48 AM, Craig Ringer wrote: >>> Storing an epoch implies that rows can't have (xmin,xmax) different by >>> more than one epoch. So if you're updating/delet

Re: [HACKERS] postgres_fdw cost estimation defaults and documentation

2017-06-05 Thread Ashutosh Bapat
mote table — but if the remote table is frequently updated, the local statistics will soon be obsolete. -- The paragraph you quoted should be read in the context of "When use_remote_estimate is false, ". It just says what happens when use_remote_estimate is turned off. It doesn't impl

Re: [HACKERS] Adding support for Default partition in partitioning

2017-06-06 Thread Ashutosh Bapat
;> COMPLETE_WITH_LIST2("FROM (", "IN ("); >> >> @@ -2483,7 +2483,7 @@ psql_completion(const char *text, int start, int >> end) >> COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_partitioned_tables, >> ""); >>

Re: [HACKERS] Adding support for Default partition in partitioning

2017-06-07 Thread Ashutosh Bapat
adding an object violate integrity constraint. The nearest errorcode seems to be ERRCODE_INVALID_OBJECT_DEFINITION, which is also used for partitions with overlapping bounds. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] A bug in mapping attributes in ATExecAttachPartition()

2017-06-07 Thread Ashutosh Bapat
I didn't get time to actually construct a testcase and test it. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Adding support for Default partition in partitioning

2017-06-08 Thread Ashutosh Bapat
On Wed, Jun 7, 2017 at 2:08 AM, Jeevan Ladhe wrote: > >> >> The code in check_default_allows_bound() to check whether the default >> partition >> has any rows that would fit new partition looks quite similar to the code >> in >> ATExecAttachPartition() checking whether all rows in the table being

Re: [HACKERS] A bug in mapping attributes in ATExecAttachPartition()

2017-06-08 Thread Ashutosh Bapat
On Thu, Jun 8, 2017 at 3:13 PM, Amit Langote wrote: > On 2017/06/08 1:44, Robert Haas wrote: >> On Wed, Jun 7, 2017 at 7:47 AM, Ashutosh Bapat >> wrote: >>> In ATExecAttachPartition() there's following code >>> >>> 13715 partnatts = get_par

Re: [HACKERS] Adding support for Default partition in partitioning

2017-06-08 Thread Ashutosh Bapat
On Thu, Jun 8, 2017 at 2:54 PM, Ashutosh Bapat wrote: > On Wed, Jun 7, 2017 at 2:08 AM, Jeevan Ladhe > wrote: > >> >>> >>> The code in check_default_allows_bound() to check whether the default >>> partition >>> has any rows that would fit n

Re: [HACKERS] Parallel Aggregation support for aggregate functions that use transitions not implemented for array_agg

2017-06-08 Thread Ashutosh Bapat
array_agg/string_agg is not a lot wider but the underlying scans are large e.g. having clause containing another aggregate and very small group sizes. I am not sure how frequent are such usecases. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sen

Re: [HACKERS] A bug in mapping attributes in ATExecAttachPartition()

2017-06-09 Thread Ashutosh Bapat
On Fri, Jun 9, 2017 at 10:31 AM, Amit Langote wrote: > On 2017/06/08 19:25, Ashutosh Bapat wrote: >> On Thu, Jun 8, 2017 at 3:13 PM, Amit Langote >>>> I think this code could be removed entirely in light of commit >>>> 3ec76ff1f2cf52e9b900349957b42d28128b7bc7. >

Re: [HACKERS] A bug in mapping attributes in ATExecAttachPartition()

2017-06-09 Thread Ashutosh Bapat
predicate_implied_by() we may want to change it under a flag. On Fri, Jun 9, 2017 at 11:43 AM, Amit Langote wrote: > On 2017/06/08 18:43, Amit Langote wrote: >> On 2017/06/08 1:44, Robert Haas wrote: >>> On Wed, Jun 7, 2017 at 7:47 AM, Ashutosh Bapat >>> wrote: >>>> In AT

Re: [HACKERS] Adding support for Default partition in partitioning

2017-06-11 Thread Ashutosh Bapat
ew partition constraint for default partition \"%s\" would be violated by > some row" > to > "updated partition constraint for default partition \"%s\" would be violated > by some row" > > Some other cosmetic changes. > > Apart from this, I am exp

Re: [HACKERS] PG10 Partitioned tables and relation_is_updatable()

2017-06-12 Thread Ashutosh Bapat
f the rewriter code > (prompted by the recent similar omission for RLS on partitioned > tables) and I didn't find any more problems there, but I haven't > looked elsewhere yet. > Changes look good to me. In order to avoid such instances in future, I have proposed to bundle the c

[HACKERS] Dropping partitioned table drops a previously detached partition

2017-06-12 Thread Ashutosh Bapat
ons. There might be more such places, but I haven't searched for those. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company From 97a647c487761782340d7ebca82b6d3200d79142 Mon Sep 17 00:00:00 2001 From: Ashutosh Bapat Date: Mon, 12 Jun 2017 16:32:53 +0

Re: [HACKERS] remove unnecessary flag has_null from PartitionBoundInfoData

2017-06-12 Thread Ashutosh Bapat
partition_bound() and get_partition_for_tuple(). Both of those callers are calling it only in case of LIST partition. So, having null_index uninitialized in PartitionBoundInfoData is not a problem. But in general, we shouldn't leave a field uninitialized in that structure, so +1 for the pat

Re: [HACKERS] PG10 Partitioned tables and relation_is_updatable()

2017-06-12 Thread Ashutosh Bapat
ables) and I didn't find any more problems there, but I haven't >>> looked elsewhere yet. > > As he mentioned in his reply, Ashutosh's proposal to abstract away the > relkind checks is interesting in this regard. > > On 2017/06/12 17:29, Ashutosh Bapat wrot

Re: [HACKERS] PG10 Partitioned tables and relation_is_updatable()

2017-06-12 Thread Ashutosh Bapat
;>>> out for more of the same. I did a quick scan of the rewriter code >>>>> (prompted by the recent similar omission for RLS on partitioned >>>>> tables) and I didn't find any more problems there, but I haven't >>>>> looked elsewhe

Re: [HACKERS] PG10 Partitioned tables and relation_is_updatable()

2017-06-12 Thread Ashutosh Bapat
I'll push my original patch and work up patches > for the other couple of issues I found. No objections, the patch is good to go as is. Sorry for high-jacking this thread. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Dropping partitioned table drops a previously detached partition

2017-06-13 Thread Ashutosh Bapat
1-Dependency-between-partitioned-table-and-partition_v1.patch. Please > find attached the v2 patch. > > > > On Tue, Jun 13, 2017 at 10:25 AM, Amit Langote > wrote: >> >> Hi Ashutosh, >> >> On 2017/06/12 20:56, Ashutosh Bapat wrote: >> > Hi, &

Re: [HACKERS] Dropping partitioned table drops a previously detached partition

2017-06-13 Thread Ashutosh Bapat
TE_TYPE_DEPENDENCY isn't conveying the real sense. But that's not a topic for this thread. I will start a separate a thread. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] Dropping partitioned table drops a previously detached partition

2017-06-13 Thread Ashutosh Bapat
On Wed, Jun 14, 2017 at 10:21 AM, Ashutosh Bapat wrote: > On Tue, Jun 13, 2017 at 7:14 PM, Rahila Syed wrote: >> I reviewed and tested >> 0001-Dependency-between-partitioned-table-and-partition_v1.patch >> It applies cleanly on master and make check passes. >> &g

Re: [HACKERS] A bug in mapping attributes in ATExecAttachPartition()

2017-06-14 Thread Ashutosh Bapat
_is_check. 0003 addresses comments by Amit Langote. In your original patch, if restrictinfo_is_check is true, we will call operator_predicate_proof(), which does not handle anything other than an operator expression. So, it's going to return false from that function. Looking at the way argi

Re: [HACKERS] A bug in mapping attributes in ATExecAttachPartition()

2017-06-14 Thread Ashutosh Bapat
NT check_a CHECK (a IS NOT NULL AND a = 6) +); +ALTER TABLE part_6 DROP c; -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] A bug in mapping attributes in ATExecAttachPartition()

2017-06-15 Thread Ashutosh Bapat
On Thu, Jun 15, 2017 at 10:46 AM, Amit Langote wrote: > Thanks for taking a look. > > On 2017/06/14 20:06, Ashutosh Bapat wrote: >> On Wed, Jun 14, 2017 at 9:20 AM, Amit Langote >> wrote: >>> >>> By the way, I mentioned an existing problem in one of the

Re: [HACKERS] A bug in mapping attributes in ATExecAttachPartition()

2017-06-15 Thread Ashutosh Bapat
On Thu, Jun 15, 2017 at 2:12 PM, Amit Langote wrote: > Thanks for the review. > > On 2017/06/15 16:08, Ashutosh Bapat wrote: >> On Thu, Jun 15, 2017 at 10:46 AM, Amit Langote wrote: >>> If we end up having to perform the validation scan and the table being >>> at

Re: [HACKERS] A bug in mapping attributes in ATExecAttachPartition()

2017-06-15 Thread Ashutosh Bapat
On Thu, Jun 15, 2017 at 2:30 PM, Amit Langote wrote: > On 2017/06/15 17:53, Ashutosh Bapat wrote: >> On Thu, Jun 15, 2017 at 2:12 PM, Amit Langote wrote: >>>> Both of the above comments are not related to the bug that is being fixed, >>>> but >>>> the

Re: [HACKERS] Adding support for Default partition in partitioning

2017-06-15 Thread Ashutosh Bapat
quot;, cannot > add a new partition", > > This error message style seems novel to me. I'm not sure about the best > message text here, but maybe: "cannot add new partition to table \"%s\" > with default partition" > > Note that the comment applies to both

Re: [HACKERS] Adding support for Default partition in partitioning

2017-06-15 Thread Ashutosh Bapat
On Fri, Jun 16, 2017 at 12:48 AM, Robert Haas wrote: > On Thu, Jun 15, 2017 at 12:54 PM, Ashutosh Bapat > wrote: >> Some more comments on the latest set of patches. >> >> In heap_drop_with_catalog(), we heap_open() the parent table to get the >> default partitio

Re: [HACKERS] [PATCH] Push limit to sort through a subquery

2017-08-23 Thread Ashutosh Bapat
n OFFSET clause even when it's possible to push it down. If we could defer deparsing the query to execution time we might be able to achieve both the targets. It has one more advantage: we could pass parameters embedded in the query, rather than binding them. -- Best Wishes, Ashut

Re: [HACKERS] path toward faster partition pruning

2017-08-23 Thread Ashutosh Bapat
ady recursive, so if we process expansion and pruning for one level in those functions, the recursion will automatically take care of doing so for every level. > > > Note that this is still largely a WIP patch and the implementation details > might change per both the feedback here and

Re: [HACKERS] Adding support for Default partition in partitioning

2017-08-29 Thread Ashutosh Bapat
onConstraints() to scan the relation or add to the queue based on some input flag may be wqueue argument itself. But I haven't thought through this completely. Any thoughts? >> >> 0004: >> Extend default partitioning support to allow addition of new partitions >> after &

Re: [HACKERS] expanding inheritance in partition bound order

2017-08-30 Thread Ashutosh Bapat
as their OIDs would be returned by RelationGetPartitionDispatchInfo(). I have a feeling that, if we go that route, we will replace almost all the changes that Amit Langote's patches do to expand_inherited_rtentry(). -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Databa

Re: [HACKERS] expanding inheritance in partition bound order

2017-08-30 Thread Ashutosh Bapat
tchInfo at all. Maybe that's actually a > better plan anyway, because as Ashutosh points out, we don't want the > partitioned children to show up before the unpartitioned children in > the resulting ordering. > +1. I think we should just pull out the OIDs from partition descripto

Re: [HACKERS] expanding inheritance in partition bound order

2017-08-30 Thread Ashutosh Bapat
On Thu, Aug 31, 2017 at 1:15 AM, Robert Haas wrote: > On Wed, Aug 30, 2017 at 12:47 PM, Ashutosh Bapat > wrote: >> +1. I think we should just pull out the OIDs from partition descriptor. > > Like this? The first patch refactors the expansion of a single child > out into a s

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2017-09-01 Thread Ashutosh Bapat
PFA the patches rebased on the latest sources. There are also fixes for some of the crashes and bugs reported. I haven't yet included the testcase patch in the main patchset. On Mon, Aug 28, 2017 at 12:44 PM, Rajkumar Raghuwanshi wrote: > On Mon, Aug 21, 2017 at 12:43 PM, Ashutosh Bapat

Re: [HACKERS] dropping partitioned tables without CASCADE

2017-09-03 Thread Ashutosh Bapat
Hi, Thomas's application to track patches told me that this patch needs rebase. It also required some changes to the code. Here's the updated version. I have squashed those two patches together. On Tue, Mar 14, 2017 at 6:35 PM, Ashutosh Bapat wrote: > Added this to 2017/7 commitf

Re: [HACKERS] dropping partitioned tables without CASCADE

2017-09-04 Thread Ashutosh Bapat
On Mon, Sep 4, 2017 at 10:34 AM, Amit Langote wrote: > Hi Ashutosh, > > On 2017/09/04 13:51, Ashutosh Bapat wrote: >> Hi, >> Thomas's application to track patches told me that this patch needs >> rebase. It also required some changes to the code. Here's th

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-04 Thread Ashutosh Bapat
On Mon, Sep 4, 2017 at 10:04 AM, Amit Langote wrote: > On 2017/09/04 12:38, Etsuro Fujita wrote: >> On 2017/09/02 4:10, Ashutosh Bapat wrote: >>> This rebase mainly changes patch 0001, which translates partition >>> hierarchy into inheritance hierarchy creating AppendR

Re: [HACKERS] dropping partitioned tables without CASCADE

2017-09-04 Thread Ashutosh Bapat
translatability. I think the > original is fine. > We have used this style in the "else" case of if (!verbose). So, I just copied it. I have removed that change in the attached patch. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company From c0a153

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-04 Thread Ashutosh Bapat
e partition-pruning > patch, because the other code in that patch relies on the same and I know > Ashuotsh has wanted that for a long time. :) Those changes are already part of my updated 0001 patch. Aren't they? May be you should just review those and see if those are suitable for you

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-04 Thread Ashutosh Bapat
On Tue, Sep 5, 2017 at 12:06 PM, Amit Langote wrote: > On 2017/09/05 15:30, Ashutosh Bapat wrote: >> Those changes are already part of my updated 0001 patch. Aren't they? >> May be you should just review those and see if those are suitable for >> you? > > Yeah, I

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-05 Thread Ashutosh Bapat
On Tue, Sep 5, 2017 at 1:16 PM, Amit Langote wrote: > On 2017/09/05 15:43, Ashutosh Bapat wrote: >> Ok. Can you please answer my previous questions? >> >> AFAIU, the list contained RTIs of the relations, which didnt' have >> corresponding AppendRelInfos to l

Re: [HACKERS] Replacing lfirst() with lfirst_node() appropriately in planner.c

2017-09-05 Thread Ashutosh Bapat
2a54a03ec90206c7e5579a562a121c Mon Sep 17 00:00:00 2001 From: Ashutosh Bapat Date: Tue, 5 Sep 2017 16:50:58 +0530 Subject: [PATCH 1/2] Use lfirst_node() instead of lfirst() wherever suitable in planner.c Ashutosh Bapat, reviewed by Jeevan Chalke --- src/backend/optimizer/plan/planner.c | 94

Re: [HACKERS] no test coverage for ALTER FOREIGN DATA WRAPPER name HANDLER ...

2017-09-05 Thread Ashutosh Bapat
only see one small oversight, which is >> the >> successful case of ALTER FOREIGN DATA WRAPPER ... HANDLER ... is still >> not tested. I added one line for that in the attached modification of your >> patch. > > Ah right, thanks. > > Adding this

Re: [HACKERS] Replacing lfirst() with lfirst_node() appropriately in planner.c

2017-09-05 Thread Ashutosh Bapat
On Wed, Sep 6, 2017 at 1:32 AM, Tom Lane wrote: > Ashutosh Bapat writes: >> On Tue, Sep 5, 2017 at 4:02 PM, Jeevan Chalke >> wrote: >>> Attached patch for replacing linitial() with linital_node() appropriately in >>> planner.c > >> Ok. The patch lo

Re: [HACKERS] paths in partitions of a dummy partitioned table

2017-09-06 Thread Ashutosh Bapat
On Fri, Aug 25, 2017 at 10:46 PM, Robert Haas wrote: > On Thu, Jul 6, 2017 at 11:35 AM, Ashutosh Bapat > wrote: >> If a partitioned table is proven dummy, set_rel_pathlist() doesn't mark the >> partition relations dummy and thus doesn't set any (dummy) paths in the

Re: [HACKERS] dropping partitioned tables without CASCADE

2017-09-06 Thread Ashutosh Bapat
27;s a partitioned table. So can you > directly > add the "is partitioned" in place of "has partitions"? > > Did those change in the attached patch and update regression expected > output. > Looks better. > Also run pgindent on the patch. > Thanks fo

Re: [HACKERS] <> join selectivity estimate question

2017-09-06 Thread Ashutosh Bapat
; > Please find attached a new version, and a test script I used, which > shows a bunch of interesting cases. I'll add this to the commitfest. I added some "stable" tests to your patch taking inspiration from the test SQL file. I think those will be stable across machines and

Re: [HACKERS] WIP: Aggregation push-down

2017-09-07 Thread Ashutosh Bapat
aggregates. 2. Following query throws error with these patches, but not without the patches. explain verbose select sum(t1.a + t2.a) from t1, t2, t2 t3 where t1.a = t2.a group by t2.a, t1.a; ERROR: ORDER/GROUP BY expression not found in targetlist [1] https://www.postgresql.org/message-id/CAFj

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-07 Thread Ashutosh Bapat
On Thu, Sep 7, 2017 at 4:32 PM, Antonin Houska wrote: > Ashutosh Bapat wrote: > >> On Fri, Sep 1, 2017 at 6:05 PM, Antonin Houska wrote: >> > >> > >> > >> > * get_partitioned_child_rels_for_join() >> > >> > I think the Assert

Re: [HACKERS] Adding support for Default partition in partitioning

2017-09-07 Thread Ashutosh Bapat
ed only from DefineRelation(), > and not for alter command. I am not really sure how can we use > work queue for create command. No, we shouldn't use work queue for CREATE command. We should extract the common code into a function to be called from check_default_allows_bound() and ValidatePartitionConstraints(). To that function we pass a flag (or the work queue argument itself), which decides whether to add a work queue item or scan the relation directly. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-08 Thread Ashutosh Bapat
On Fri, Sep 8, 2017 at 12:34 AM, Robert Haas wrote: > On Tue, Sep 5, 2017 at 7:01 AM, Ashutosh Bapat > wrote: >> accumulate_append_subpath() is executed for every path instead of >> every relation, so changing it would collect the same list multiple >> times. Instead

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-11 Thread Ashutosh Bapat
On Sat, Sep 9, 2017 at 6:28 AM, Robert Haas wrote: > On Fri, Sep 8, 2017 at 1:38 PM, Ashutosh Bapat > wrote: >>> I also confirmed that the partition-pruning patch set works fine with this >>> patch instead of the patch on that thread with the same functionality, >&g

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2017-09-11 Thread Ashutosh Bapat
On Thu, Sep 7, 2017 at 7:34 PM, Antonin Houska wrote: > Ashutosh Bapat wrote: > >> I have fixed the issues which were marked as TODOs in the attached >> patches. Also, I have included your test change patch in my series of >> patches. > > I've noticed that

Re: [HACKERS] [POC] hash partitioning

2017-09-11 Thread Ashutosh Bapat
against this commit & renamed to 0001, PFA. Given that we have default partition support now, I am wondering whether hash partitioned tables also should have default partitions. The way we have structured hash partitioning syntax, there can be "holes" in partitions. Default part

Re: [HACKERS] WIP: Aggregation push-down

2017-09-11 Thread Ashutosh Bapat
On Fri, Sep 8, 2017 at 7:04 PM, Antonin Houska wrote: > Ashutosh Bapat wrote: > >> On Thu, Aug 17, 2017 at 8:52 PM, Antonin Houska wrote: >> > Antonin Houska wrote: >> > >> >> Antonin Houska wrote: >> >> >> >> > This

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-11 Thread Ashutosh Bapat
On Mon, Sep 11, 2017 at 12:16 PM, Amit Langote wrote: > On 2017/09/09 2:38, Ashutosh Bapat wrote: >> On Fri, Sep 8, 2017 at 11:17 AM, Amit Langote wrote: >>> I updated the patch to include just those changes. I'm not sure about >>> one of the Ashutosh's cha

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-11 Thread Ashutosh Bapat
On Mon, Sep 11, 2017 at 2:16 PM, Amit Langote wrote: > On 2017/09/11 16:23, Ashutosh Bapat wrote: >> On Sat, Sep 9, 2017 at 6:28 AM, Robert Haas wrote: >>> I'm a bit suspicious about the fact that there are now executor >>> changes related to the PlanRowMarks.

Re: [HACKERS] Automatic testing of patches in commit fest

2017-09-11 Thread Ashutosh Bapat
> with automatic messages to often. I believe that sending such messages > once a day would be enough. > > Unless there are any objections to give this idea a try I'm willing to > write and host a corresponding script. > +1 that would help. -- Best Wishes, Ashutosh Bapat E

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-11 Thread Ashutosh Bapat
On Mon, Sep 11, 2017 at 5:19 PM, Robert Haas wrote: > On Mon, Sep 11, 2017 at 6:45 AM, Ashutosh Bapat > wrote: >> So, all partitioned partitions are getting locked correctly. Am I >> missing something? > > That's not a valid test. In that scenario, you're goin

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-12 Thread Ashutosh Bapat
On Tue, Sep 12, 2017 at 7:31 AM, Amit Langote wrote: > On 2017/09/11 19:45, Ashutosh Bapat wrote: >> On Mon, Sep 11, 2017 at 12:16 PM, Amit Langote wrote: >>> IMHO, we should make it the responsibility of the future patch to set a >>> child PlanRowMark's prti to th

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-12 Thread Ashutosh Bapat
On Tue, Sep 12, 2017 at 1:16 PM, Amit Langote wrote: > On 2017/09/11 21:07, Ashutosh Bapat wrote: >> On Mon, Sep 11, 2017 at 5:19 PM, Robert Haas wrote: >>> On Mon, Sep 11, 2017 at 6:45 AM, Ashutosh Bapat >>> wrote: >>>> So, all partitioned partit

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-12 Thread Ashutosh Bapat
On Tue, Sep 12, 2017 at 1:42 PM, Amit Langote wrote: > On 2017/09/12 16:55, Ashutosh Bapat wrote: >> On Tue, Sep 12, 2017 at 1:16 PM, Amit Langote wrote: >>> So I looked at this a bit closely and came to the conclusion that we may >>> not need to keep partitio

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-12 Thread Ashutosh Bapat
/* Include child's rowmark type in parent's allMarkTypes */ parentrc->allMarkTypes |= childrc->allMarkTypes; } so that we push allMarkTypes up the hierarchy. I like the second way, since every intermediate parent records allMarkTypes of its descendants.

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-12 Thread Ashutosh Bapat
On Tue, Sep 12, 2017 at 2:35 PM, Amit Langote wrote: > On 2017/09/12 17:53, Ashutosh Bapat wrote: >> On Tue, Sep 12, 2017 at 1:42 PM, Amit Langote wrote: >>> So, we can remove partitioned_rels from (Merge)AppendPath and >>> (Merge)Append nodes and remove ExecLockNonLeaf

Re: [HACKERS] no test coverage for ALTER FOREIGN DATA WRAPPER name HANDLER ...

2017-09-12 Thread Ashutosh Bapat
On Tue, Sep 12, 2017 at 2:27 PM, Amit Langote wrote: > Thanks Ashutosh for taking a look at this. > > On 2017/09/05 21:16, Ashutosh Bapat wrote: >> The patch needs a rebase. > > Attached rebased patch. Thanks for rebased patch. We could annotate each ERROR with an explana

Re: [HACKERS] Log LDAP "diagnostic messages"?

2017-09-12 Thread Ashutosh Bapat
The patch needs to address some comments in the previous mails, so marking it as "waiting for author". On Wed, Aug 16, 2017 at 11:13 AM, Ashutosh Bapat wrote: > On Wed, Aug 16, 2017 at 8:44 AM, Alvaro Herrera > wrote: >> Christoph Berg wrote: >>> Re: Thomas Munro

Re: [HACKERS] dropping partitioned tables without CASCADE

2017-09-12 Thread Ashutosh Bapat
for rebasing and improving the patch. Looks >> good to me too. > > Patch needed to be rebased after the default partitions patch went in, so > done. Per build status on http://commitfest.cputube.org :) > > Thanks, > Amit -- Best Wishes, Ashutosh Bapat EnterpriseDB Co

Re: [HACKERS] Constraint exclusion for partitioned tables

2017-09-13 Thread Ashutosh Bapat
exclusion always. For a partitioned table, this patch saves the time to run constraint exclusion on all the partitions if constraint exclusion succeeds on the partitioned table. If constraint exclusion fails, we have wasted CPU cycles on one run of constraint exclusion. The difference between the

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-13 Thread Ashutosh Bapat
ts? -- Amit Langote agrees with this. It kind of makes the assertion lame but keeps the code sane. What do you think? [1] https://www.postgresql.org/message-id/d2f1cdcb-ebb4-76c5-e471-79348ca5d...@lab.ntt.co.jp [2] https://www.postgresql.org/message-id/CAFjFpRfJ3GRRmmOugaMA-q4i=se5p6yjz_c6a6hdrdqqtgx...@mail.gmail.com -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-13 Thread Ashutosh Bapat
On Wed, Sep 13, 2017 at 11:29 AM, Amit Langote wrote: > On 2017/09/12 19:56, Ashutosh Bapat wrote: >> I think the code here expects the original parent_rte and not the one >> we set around line 1169. >> >> This isn't a bug right now, since both the parent_rte s ha

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-13 Thread Ashutosh Bapat
t; > All the TPCH results are posted in the same above mail thread. Can you please check if the attached patch fixes the issue. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company commit 203b3083318e9da41ad614a2ccec532025877c3b Author: Ashutosh Bapat Date:

Re: [HACKERS] no test coverage for ALTER FOREIGN DATA WRAPPER name HANDLER ...

2017-09-13 Thread Ashutosh Bapat
On Wed, Sep 13, 2017 at 7:49 AM, Amit Langote wrote: > On 2017/09/12 20:17, Ashutosh Bapat wrote: >> On Tue, Sep 12, 2017 at 2:27 PM, Amit Langote >> wrote: >>> Thanks Ashutosh for taking a look at this. >>> >>> On 2017/09/05 21:16, Ashutosh

Re: [HACKERS] no test coverage for ALTER FOREIGN DATA WRAPPER name HANDLER ...

2017-09-13 Thread Ashutosh Bapat
On Wed, Sep 13, 2017 at 1:27 PM, Amit Langote wrote: > On 2017/09/13 16:42, Ashutosh Bapat wrote: >> On Wed, Sep 13, 2017 at 7:49 AM, Amit Langote wrote: >>> In the attached updated patch, I created separate .source files in >>> src/test/regress/input an

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-13 Thread Ashutosh Bapat
On Wed, Sep 13, 2017 at 12:51 PM, Ashutosh Bapat wrote: > On Wed, Sep 13, 2017 at 12:39 AM, Robert Haas wrote: >> On Tue, Sep 12, 2017 at 3:46 AM, Amit Langote >> wrote: >>> In this case, AcquireExecutorLocks will lock all the relations in >>> PlannedSt

Re: [HACKERS] <> join selectivity estimate question

2017-09-13 Thread Ashutosh Bapat
On Thu, Sep 14, 2017 at 4:19 AM, Thomas Munro wrote: > On Wed, Sep 6, 2017 at 11:14 PM, Ashutosh Bapat > wrote: >> On Fri, Jul 21, 2017 at 4:10 AM, Thomas Munro >> wrote: >>> That just leaves the question of whether we should try to handle the >>> empty

Re: [HACKERS] <> join selectivity estimate question

2017-09-13 Thread Ashutosh Bapat
On Thu, Sep 14, 2017 at 4:30 AM, Tom Lane wrote: > Thomas Munro writes: >> On Wed, Sep 6, 2017 at 11:14 PM, Ashutosh Bapat >> wrote: >>> I added some "stable" tests to your patch taking inspiration from the >>> test SQL file. I think those will be stab

Re: [HACKERS] Log LDAP "diagnostic messages"?

2017-09-14 Thread Ashutosh Bapat
On Wed, Sep 13, 2017 at 6:58 AM, Thomas Munro wrote: > On Tue, Sep 12, 2017 at 11:23 PM, Ashutosh Bapat > wrote: >> On Wed, Aug 16, 2017 at 11:13 AM, Ashutosh Bapat >> wrote: >>> On Wed, Aug 16, 2017 at 8:44 AM, Alvaro Herrera >>> wrote: >>>>

Re: [HACKERS] no test coverage for ALTER FOREIGN DATA WRAPPER name HANDLER ...

2017-09-14 Thread Ashutosh Bapat
lright, done. Thanks for reviewing. > LGTM. The patch applies cleanly on the current HEAD, compiles (small bit in regress.c requires compilation), and make check passes. Marking this as "ready for committer". -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Datab

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-15 Thread Ashutosh Bapat
/CAFjFpRedUZPa7tKbCLEGK3u5UWdDNQoN=eyfb7ieg5d0d1p...@mail.gmail.com [3] https://www.postgresql.org/message-id/cafjfprejksdcfaeuzjgd79hoetzpz5bkdxljgxr7qznrxx+...@mail.gmail.com -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Re: [COMMITTERS] pgsql: Expand partitioned table RTEs level by level, without flattening

2017-09-18 Thread Ashutosh Bapat
true. The array can contain "dead" relations as well. I have removed the assertion and instead fixed the code to skip anything which is not "base" or "other member rel". I have also added a test to cover dead relations and lateral references in join.sql. -- Best Wishe

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-18 Thread Ashutosh Bapat
On Sat, Sep 16, 2017 at 2:53 AM, Robert Haas wrote: > On Fri, Sep 15, 2017 at 6:11 AM, Ashutosh Bapat > wrote: >> Thanks a lot Robert. >> >> Here are rebased patches. > > I didn't get quite as much time to look at these today as I would have > liked, but her

Re: [HACKERS] postgres_fdw bug in 9.6

2017-09-19 Thread Ashutosh Bapat
On Tue, Sep 5, 2017 at 1:10 PM, Etsuro Fujita wrote: > > > I think Tom is reviewing this patch [1]. > I am marking this as ready for committer as I don't have any new comments and possibly other reviewers have also done reviewing it. -- Best Wishes, Ashutosh Bapat EnterpriseD

Re: [HACKERS] PoC: full merge join on comparison clause

2017-09-19 Thread Ashutosh Bapat
ot add {} around a single statement block. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Varying results when using merge joins over postgres_fdw vs hash joins

2017-09-20 Thread Ashutosh Bapat
_en_us_utf8_encoding e > (cost=100.00..110.67 rows=1 width=1548) (actual time=8.289..144.210 > rows=33418 loops=1) >Output: e.str1, e.str2, e.str3 >Remote SQL: SELECT str1, str2, str3 FROM > public.table_with_en_us_utf8_encoding WHERE ((str4 = '2'::text)) > Planning time: 0.153 ms > Execution time: 156.557 ms > (13 rows) > > In this case, both tables use same collation while comparing the rows, so result is different from the merge join result. Hash join executed on local server and the same executed on foreign server (by importing local table to the foreign server) would also differ. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-20 Thread Ashutosh Bapat
ningful out of this. May be there's some faster way by looking at the lines that are covered by B but not A. BTW, I checked those lines to see if there could be any bug there. But I don't see what could go wrong with those lines. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation T

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-20 Thread Ashutosh Bapat
On Tue, Sep 19, 2017 at 3:17 PM, Ashutosh Bapat wrote: >> >>>> - I'm not entirely sure whether maintaining partexprs and >>>> nullable_partexprs is the right design. If I understand correctly, >>>> whether or not a partexpr is nullable is reall

Re: [HACKERS] [Proposal] Make the optimiser aware of partitions ordering

2017-09-21 Thread Ashutosh Bapat
the sort keys match partition keys by creating an Append path by picking sorted paths from partition RelOptInfos in order. You may use slightly modified version of add_paths_to_append_rel(). -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-21 Thread Ashutosh Bapat
Worker 3: actual time=0.927..3130.497 rows=475545 loops=1 If we sum the rows returned by each worker they don't add up to (actual rows) * (actual loops). So I assumed that the unreported number of rows were processed by the leader. Is that right? I might be m

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-22 Thread Ashutosh Bapat
when the tables are partitioned. Am I interpreting the results correctly? There may be some other way of partitioning, which may give better results, but I think what we have now shows the importance of partitioning in case of very large data e.g. scale 300 TPCH. -- Best Wishes, Ashutosh Bapat E

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-22 Thread Ashutosh Bapat
On Fri, Sep 15, 2017 at 5:29 PM, Ashutosh Bapat wrote: > >> >> Apart from these there is a regression case on a custom table, on head >> query completes in 20s and with this patch it takes 27s. Please find >> the attached .out and .sql file for the output and

Re: [HACKERS] Partition-wise aggregation/grouping

2017-09-26 Thread Ashutosh Bapat
- > Jeevan Chalke > Principal Software Engineer, Product Development > EnterpriseDB Corporation > The Enterprise PostgreSQL Company > > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > ht

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2017-09-27 Thread Ashutosh Bapat
remote server. But even if that's not the >> scenario, I think we should assume that the DBA or some other system >> resolved it and therefore we don't need to do anything further. If we >> assume anything else, then we just go into an infinite error loop, >> w

Re: [HACKERS] PartitionSchemaData & partcollation (Re: [COMMITTERS] pgsql: Associate partitioning information with each RelOptInfo.)

2017-09-28 Thread Ashutosh Bapat
r partition pruning without constraint >> exclusion. >> >> Ashutosh Bapat, reviewed by Amit Langote and with quite a few changes, >> mostly cosmetic, by me. Additional review and testing of this patch >> series by Antonin Houska, Amit Khandekar, Rafia Sabih, Rajkumar >

Re: [HACKERS] Partition-wise aggregation/grouping

2017-09-28 Thread Ashutosh Bapat
On Wed, Sep 27, 2017 at 3:42 PM, Jeevan Chalke wrote: > Thanks Ashutosh for reviewing. > > Attached new patch-set with following changes: > > 1. Removed earlier 0007 and 0008 patches which were PoC for supporting > partial aggregation over fdw. I removed them as it will be a different > issue alto

Re: [HACKERS] PoC: full merge join on comparison clause

2017-10-02 Thread Ashutosh Bapat
al_cost_mergejoin(). Not sure if that > approximation makes any sense, but this is the best I could think of. > > Style problems are fixed. > > Attached please find the new version of the patch that addresses all the > review comments except mergeopfamilies. > > The current co

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2017-10-02 Thread Ashutosh Bapat
want to do. > +1. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Another oddity in handling of WCO constraints in postgres_fdw

2017-10-03 Thread Ashutosh Bapat
ks odd. May be we should just leave this case as it is. I am comparing this case with the way we handle constraints on a foreign table. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2015-07-20 Thread Ashutosh Bapat
mpare them by OID. But IMO creating > UserMapping for every RelOptInfo seems waste of space and time. > > Thoughts? > -- > Shigeru HANADA > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers > > -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

<    1   2   3   4   5   6   7   8   9   10   >