Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-04-15 Thread Ashutosh Bapat
On Fri, Apr 15, 2016 at 9:39 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Thu, Apr 14, 2016 at 7:49 AM, Ashutosh Bapat > <ashutosh.ba...@enterprisedb.com> wrote: > > BTW, I noticed that we are deparsing whole-row reference as ROW(list of > > columns from loc

Re: [HACKERS] Declarative partitioning

2016-04-15 Thread Ashutosh Bapat
onDescData as { int nparts; PartitionInfo *partinfo; /* array of partition information structures. */ } The new syntax allows CREATE TABLE to be specified as partition of an already partitioned table. Is it possible to do the same for CREATE FOREIGN TABLE? Or that's material for v2? Similarly for ATTACH PARTITION. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Declarative partitioning

2016-04-15 Thread Ashutosh Bapat
ATTACH PARTITION. 3. PartitionKeyData contains KeyTypeCollInfo, whose contents can be obtained by calling functions exprType, exprTypemod on partexprs. Why do we need to store that information as a separate member? -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Declarative partitioning

2016-04-14 Thread Ashutosh Bapat
forward to working on this for 9.7. > > Thanks, > Amit > > > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers > -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-04-14 Thread Ashutosh Bapat
END expression. PFA patch with that fix included and also some testcases for system columns as well as whole-row references. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c index bdc4

Re: [HACKERS] Postgres_fdw join pushdown - INNER - FULL OUTER join combination generating wrong result

2016-04-14 Thread Ashutosh Bapat
On Thu, Apr 14, 2016 at 8:42 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: > On 2016/03/29 23:20, Ashutosh Bapat wrote: > >> I think the reason for that is in foreign_join_ok. This in that >> function: >> >> wrongly pulls up remote_cond

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-04-05 Thread Ashutosh Bapat
s issue should be fixed in advance of the PostgreSQL > 9.6beta1 release. > > Best regards, > Etsuro Fujita > -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Postgres_fdw join pushdown - INNER - FULL OUTER join combination generating wrong result

2016-03-29 Thread Ashutosh Bapat
hat was reported, the bug triggered because of the way conditions are handled for an inner join. For an inner join, all the conditions in ON as well as WHERE clause are treated like they are part of WHERE clause. This allows pushing down a join even if there are unpushable join clauses. But the pu

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-03-24 Thread Ashutosh Bapat
nce change in tableoid between planning and execution will trigger plan cache invalidation. I haven't tried this though. Sorry for bringing this solution late to the table. On Thu, Mar 24, 2016 at 3:04 PM, Ashutosh Bapat < ashutosh.ba...@enterprisedb.com> wrote: > > > On Thu, Mar 24

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-03-24 Thread Ashutosh Bapat
On Thu, Mar 24, 2016 at 9:31 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: > On 2016/03/23 13:44, Ashutosh Bapat wrote: > >> An FDW can choose not to use those functions, so I don't see a >> connection between scan list having simple Vars and existence of thos

Re: [HACKERS] Postgres_fdw join pushdown - getting server crash in left outer join of three table

2016-03-24 Thread Ashutosh Bapat
> > Thanks for the report and the testing. I have committed the patch. > > Thanks. > Cool, I have refreshed the wiki page for open items accordingly. > Thanks. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-03-22 Thread Ashutosh Bapat
On Wed, Mar 23, 2016 at 8:20 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: > On 2016/03/22 21:10, Ashutosh Bapat wrote: > >> On Tue, Mar 22, 2016 at 5:05 PM, Etsuro Fujita >> <fujita.ets...@lab.ntt.co.jp <mailto:fujita.ets...@lab.ntt.co.jp>> wrote:

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-03-22 Thread Ashutosh Bapat
On Tue, Mar 22, 2016 at 5:05 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: > On 2016/03/22 14:54, Ashutosh Bapat wrote: > >> On Tue, Mar 22, 2016 at 8:03 AM, Etsuro Fujita >> <fujita.ets...@lab.ntt.co.jp <mailto:fujita.ets...@lab.ntt.co.jp>> wrote:

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-03-21 Thread Ashutosh Bapat
instead, I think, core should provide functions which can be used by FDWs, if they want, to return values for those columns. Something like Datum get_syscol_value(RelOptInfo/Relation, attno). The function will return Datum 0 for most of the columns and table's OID for tableoid. My 0.02. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Postgres_fdw join pushdown - getting server crash in left outer join of three table

2016-03-21 Thread Ashutosh Bapat
add pathkeys which will help merge joins. I have included the relevant tests rewriting them to use local tables, so that the entire join is not pushed down to the foreign server. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company diff --git a/contrib/postg

Re: [HACKERS] Re: [COMMITTERS] pgsql: Only try to push down foreign joins if the user mapping OIDs mat

2016-03-19 Thread Ashutosh Bapat
On Wed, Mar 16, 2016 at 10:22 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Robert Haas <robertmh...@gmail.com> writes: > > On Wed, Mar 16, 2016 at 4:10 AM, Ashutosh Bapat < > > ashutosh.ba...@enterprisedb.com> wrote: > >> In 9.5, postgres_fdw allow

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-03-19 Thread Ashutosh Bapat
That means those attributes won't be set while fetching the row from the foreign server and will have garbage values in corresponding places. I guess that would work. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-03-18 Thread Ashutosh Bapat
ons. It seems very much possible after the upper pathification changes. We can not have a query sent to the foreign server for a relation, when pushdown_safe is false for that. Your patch does that for foreign base relation which try to fetch system columns. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Re: [COMMITTERS] pgsql: Only try to push down foreign joins if the user mapping OIDs mat

2016-03-16 Thread Ashutosh Bapat
On Wed, Mar 16, 2016 at 2:14 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Tue, Mar 15, 2016 at 6:44 AM, Ashutosh Bapat > <ashutosh.ba...@enterprisedb.com> wrote: > > Here's patch which fixes the issue using Robert's idea. > > Please at least check your

Re: [HACKERS] Re: [COMMITTERS] pgsql: Only try to push down foreign joins if the user mapping OIDs mat

2016-03-15 Thread Ashutosh Bapat
terpriseDB: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers > -- Best Wishes, Ashut

Re: [HACKERS] Re: [COMMITTERS] pgsql: Only try to push down foreign joins if the user mapping OIDs mat

2016-03-14 Thread Ashutosh Bapat
ser mapping for an FDW which specifies that user mapping is meaningless to it. Should we allow the user mapping to be created but ignore it or do not allow it to be created? In the later case, what should happen to the existing user mappings? -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Use %u to print user mapping's umid and userid

2016-03-14 Thread Ashutosh Bapat
On Mon, Mar 14, 2016 at 1:29 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: > Hi, > > On 2016/02/09 14:09, Ashutosh Bapat wrote: > >> Sorry, I was wrong. For public user mapping userid is 0 (InvalidOid), >> which is returned as is in UserMapping object. I

Re: [HACKERS] Obsolete comment in postgres_fdw.c

2016-03-14 Thread Ashutosh Bapat
h/joinpath.c. > Those look fine. Sorry for missing those in the commit and thanks for providing a patch for the same. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Adjusting the API of pull_var_clause()

2016-03-10 Thread Ashutosh Bapat
en pull_var_clause was being written afresh. Now, that we have this API, I am not sure whether the effort is worth the result. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Pushing down sorted joins

2016-03-09 Thread Ashutosh Bapat
On Wed, Mar 9, 2016 at 9:22 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Mar 9, 2016 at 2:23 AM, Ashutosh Bapat > <ashutosh.ba...@enterprisedb.com> wrote: > > [ new patch ] > > This looks OK to me. Committed! > > Thanks. -- Best Wishes, Ash

Re: [HACKERS] Pushing down sorted joins

2016-03-08 Thread Ashutosh Bapat
member A.c3 in the pathkeys and em_relids will indicate only A. Hence instead of equal, (which used to be OK for single relation join push-down) we have to use subset operation. We want an equivalence members whose relids are subset of relids contained by given relation. -- Best Wishes, Ash

Re: [HACKERS] Issue with NULLS LAST, with postgres_fdw sort pushdown

2016-03-02 Thread Ashutosh Bapat
On Thu, Mar 3, 2016 at 7:27 AM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Wed, Mar 2, 2016 at 7:04 PM, Rajkumar Raghuwanshi > <rajkumar.raghuwan...@enterprisedb.com> wrote: > > On Wed, Mar 2, 2016 at 2:35 PM, Ashutosh Bapat > > <ashut

Re: [HACKERS] Issue with NULLS LAST, with postgres_fdw sort pushdown

2016-03-02 Thread Ashutosh Bapat
sword 'db2'); > > --create a foreign table > create foreign table fdw_sort_test (id integer, name varchar(50)) server > db2_link_server; > > --run the below query and checkout the output > select * from fdw_sort_test order by name desc nulls last; > > --check the explain plan &g

Re: [HACKERS] GetExistingLocalJoinPath() vs. the docs

2016-03-01 Thread Ashutosh Bapat
n't seem to fit into this section at > all. It's really quite different from the others listed there. How > about something like the attached instead? Right. Mentioning the function in the description of relevant function looks better and avoids some duplication. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Pushing down sorted joins

2016-02-23 Thread Ashutosh Bapat
in their right place. On Wed, Feb 17, 2016 at 5:37 PM, Ashutosh Bapat < ashutosh.ba...@enterprisedb.com> wrote: > Hi All, > Now that we have join pushdown support in postgres_fdw, we can leverage > the sort pushdown mechanism for base relations to work for pushed down > joins

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-18 Thread Ashutosh Bapat
On Thu, Feb 18, 2016 at 3:48 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: > On 2016/02/16 16:40, Etsuro Fujita wrote: > >> On 2016/02/16 16:02, Ashutosh Bapat wrote: >> >>> On Tue, Feb 16, 2016 at 12:26 PM, Etsuro Fujita >>> <f

[HACKERS] Pushing down sorted joins

2016-02-17 Thread Ashutosh Bapat
-id/CAFjFpRfeKHiCmwJ72p4=zvuzrqsau9tbfyw7vwr-5ppvrcb...@mail.gmail.com -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company pg_join_sort_pd.patch Description: application/download \set num_samples 10 \set query 'SELECT lt1.val, ft1.val, ft2.val FROM lt1 join (ft1 join ft2 on (ft1.

Re: [HACKERS] GetExistingLocalJoinPath() vs. the docs

2016-02-16 Thread Ashutosh Bapat
s (it > doesn't return a void) Thanks for pointing that out. > and the paragraph about what it's for could do > with some wordsmithing. > Any specific suggestions? > > A link from 54.2 to 54.3 which mentions it would be fine, of course. > Ok PFA patch fixing those things

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-15 Thread Ashutosh Bapat
On Tue, Feb 16, 2016 at 12:26 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp > wrote: > On 2016/02/16 15:22, Ashutosh Bapat wrote: > >> During join planning, the planner tries multiple combinations of joining >> relations, thus the same base or join relation

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-15 Thread Ashutosh Bapat
<fujita.ets...@lab.ntt.co.jp > wrote: > On 2016/02/15 21:33, Ashutosh Bapat wrote: > >> Here's patch with better way to fix it. I think while concatenating the >> lists, we need to copy the lists being appended and in all the cases. If >> we don't copy, a change in

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-15 Thread Ashutosh Bapat
lists of any higher level joins. On Mon, Feb 15, 2016 at 1:10 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: > On 2016/02/10 4:16, Robert Haas wrote: > >> On Tue, Feb 9, 2016 at 8:39 AM, Ashutosh Bapat >> <ashutosh.ba...@enterprisedb.com> wrote: >&g

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-10 Thread Ashutosh Bapat
..@gmail.com> > wrote: > > On Tue, Feb 9, 2016 at 8:39 AM, Ashutosh Bapat > > <ashutosh.ba...@enterprisedb.com> wrote: > >> Thanks Jeevan for your review and comments. PFA the patch which fixes > those. > > > > Committed with a couple more small a

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-09 Thread Ashutosh Bapat
Yay, finally! Thanks. On Wed, Feb 10, 2016 at 12:46 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Tue, Feb 9, 2016 at 8:39 AM, Ashutosh Bapat > <ashutosh.ba...@enterprisedb.com> wrote: > > Thanks Jeevan for your review and comments. PFA the patch which fixe

Re: [HACKERS] [COMMITTERS] pgsql: postgres_fdw: Push down joins to remote servers.

2016-02-09 Thread Ashutosh Bapat
doesn't seem like a very good idea; I'll > remove that test. > > -- > Robert Haas > EnterpriseDB: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your su

Re: [HACKERS] Use %u to print user mapping's umid and userid

2016-02-08 Thread Ashutosh Bapat
Sorry, I was wrong. For public user mapping userid is 0 (InvalidOid), which is returned as is in UserMapping object. I confused InvalidOid with -1. Sorry for the confusion. On Tue, Feb 9, 2016 at 10:21 AM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Ashutosh Bapat <ashutosh.ba...@ente

Re: [HACKERS] Use %u to print user mapping's umid and userid

2016-02-08 Thread Ashutosh Bapat
sql.org/mailpref/pgsql-hackers > -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-05 Thread Ashutosh Bapat
all of them output the same targetlist. We don't need to make sure that targetlist match as long as we are using the targetlist passed in by create_scan_plan(). Do you have a counter example? -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-05 Thread Ashutosh Bapat
st Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c index fb72f45..7a2a67b 100644 --- a/contrib/postgres_fdw/deparse.c +++ b/contrib/postgres_fdw/deparse.c @@ -93,9 +93,10 @@ typedef struct foreig

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-04 Thread Ashutosh Bapat
wrote: >>> >>>> On Wed, Feb 3, 2016 at 12:08 PM, Ashutosh Bapat >>>> <ashutosh.ba...@enterprisedb.com> wrote: >>>> >>>>> PFA patches with naming conventions similar to previous ones. >>>>> pg_fdw_core_v7.patch: c

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-04 Thread Ashutosh Bapat
uld be closer to parameterized foreign paths (if we were to produce those). Hence my statement. There is always a possibility that those two costs are way too different, hence I have used phrase "possibly" there. I could be wrong. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-04 Thread Ashutosh Bapat
ath should be a local path set when paths for outerjoinpath->parent was being created. Am I missing something? -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-04 Thread Ashutosh Bapat
On Thu, Feb 4, 2016 at 4:30 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Feb 3, 2016 at 5:56 PM, Robert Haas <robertmh...@gmail.com> wrote: > > On Wed, Feb 3, 2016 at 12:08 PM, Ashutosh Bapat > > <ashutosh.ba...@enterprisedb.com> wrote: > >> P

Re: [HACKERS] [POC] FETCH limited by bytes.

2016-02-03 Thread Ashutosh Bapat
ng > regression > > tests. > > > > Regression tests added. > > Committed. > > -- > Robert Haas > EnterpriseDB: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-02 Thread Ashutosh Bapat
On Tue, Feb 2, 2016 at 5:18 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Mon, Feb 1, 2016 at 8:27 AM, Ashutosh Bapat > <ashutosh.ba...@enterprisedb.com> wrote: > > Here are patches rebased on recent commit > > cc592c48c58d9c1920f8e2063756dcbcce79e4dd. Renam

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-01-29 Thread Ashutosh Bapat
On Fri, Jan 29, 2016 at 2:05 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: > On 2016/01/29 1:26, Ashutosh Bapat wrote: > >> Here's an updated version of the previous patches, broken up like before >> > > 2. pg_fdw_join_v3.patch: changes to postgre

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-01-29 Thread Ashutosh Bapat
On Fri, Jan 29, 2016 at 9:51 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Thu, Jan 28, 2016 at 11:26 AM, Ashutosh Bapat > <ashutosh.ba...@enterprisedb.com> wrote: > > 2. pg_fdw_join_v3.patch: changes to postgres_fdw - more description below > > This patch n

[HACKERS] Using user mapping OID as hash key for connection hash

2016-01-27 Thread Ashutosh Bapat
pbq+lgpxtchnuqfobjg2...@mail.gmail.com -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company pg_fdw_concache.patch.large Description: Binary data pg_fdw_concache.patch.short Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-01-21 Thread Ashutosh Bapat
alid user id for public user mapping) and used userid from that structure, we will get rid of this problem. > > -- > Robert Haas > EnterpriseDB: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-01-21 Thread Ashutosh Bapat
On Thu, Jan 21, 2016 at 3:03 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Mon, Jan 18, 2016 at 6:47 AM, Ashutosh Bapat > <ashutosh.ba...@enterprisedb.com> wrote: > > 2. pg_fdw_join_v2.patch: postgres_fdw changes for supporting join > pushdown > > The very

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-01-20 Thread Ashutosh Bapat
h, postgresBeginForeignScan() obtained user mapping using GetUserMappingById() instead of the earlier way of fetching it by userid and serverid. So even that change will remain, right? -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-01-20 Thread Ashutosh Bapat
I missed the example plan cache revalidation patch in the previous mail. Sorry. Here it is. On Wed, Jan 20, 2016 at 7:20 PM, Ashutosh Bapat < ashutosh.ba...@enterprisedb.com> wrote: > > > On Wed, Jan 20, 2016 at 4:58 AM, Robert Haas <robertmh...@gmail.com> > wrote: > &

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-01-18 Thread Ashutosh Bapat
NNf9Z=gnyksb6wg...@mail.gmail.com I will be working next on (in that order) 1. eval_plan_qual fix for foreign join. (Considered as a must-have for 9.6) 2. Pushing down ORDER BY clause along with join pushdown 3. Parameterization of foreign join paths (Given the complexity of the feature this may not make it into 9.6) -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Limit and inherited tables

2016-01-17 Thread Ashutosh Bapat
get sorted data from the foreign server. LIMIT pushdown is not supported yet, though. > > -- > Konstantin Knizhnik > Postgres Professional: http://www.postgrespro.com > The Russian Postgres Company > > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.or

Re: [HACKERS] FDW join pushdown and scanclauses

2016-01-13 Thread Ashutosh Bapat
On Thu, Jan 14, 2016 at 9:31 AM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: > On 2016/01/08 22:05, Ashutosh Bapat wrote: > >> In add_paths_to_joinrel(), the FDW specific hook GetForeignJoinPaths() >> is called. This hook if implemented should add ForeignPaths

[HACKERS] FDW join pushdown and scanclauses

2016-01-08 Thread Ashutosh Bapat
age-id/cafjfprdhgenohm0ab6gxz1evx_yoqkywukddzeb5vpzfbae...@mail.gmail.com -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c index 953aa62..bb1c058 100644 --- a/src/b

Re: [HACKERS] use_remote_estimate usage for join pushdown in postgres_fdw

2016-01-05 Thread Ashutosh Bapat
} ; This use_remote_estimate is true if server level option is true or table level option is true. ANDing or ORing use_remote_estimate from the joining relations' PgFdwRelationInfo looks cheaper than pulling it out of ForeignServer structure. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2015-12-24 Thread Ashutosh Bapat
On Thu, Dec 24, 2015 at 8:32 AM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Mon, Nov 9, 2015 at 8:55 PM, Ashutosh Bapat > <ashutosh.ba...@enterprisedb.com> wrote: > > > > > > On Sat, Nov 7, 2015 at 12:07 AM, Robert Haas <robertmh...@gmail.com&g

Re: [HACKERS] Getting sorted data from foreign server for merge join

2015-12-22 Thread Ashutosh Bapat
Thanks. On Wed, Dec 23, 2015 at 12:24 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Mon, Dec 21, 2015 at 6:34 AM, Ashutosh Bapat > <ashutosh.ba...@enterprisedb.com> wrote: > >> I went over this patch in some detail today and did a lot of cosmetic > >>

Re: [HACKERS] Costing foreign joins in postgres_fdw

2015-12-21 Thread Ashutosh Bapat
On Fri, Dec 18, 2015 at 6:39 PM, Albe Laurenz <laurenz.a...@wien.gv.at> wrote: > Ashutosh Bapat wrote: > > Costs for foreign queries are either obtained from the foreign server > using EXPLAIN (if > > use_remote_estimate is ON) otherwise they are cooked up locally b

Re: [HACKERS] Getting sorted data from foreign server for merge join

2015-12-21 Thread Ashutosh Bapat
t be associated with a query per say and EC's are always unique because of canonicalisation. May be we should reword it as "Extract single EC for ordering of query, if any, so we don't consider it again." Is that cryptic as well? -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation Th

[HACKERS] Costing foreign joins in postgres_fdw

2015-12-18 Thread Ashutosh Bapat
resql.org/message-id/CAFjFpRepSC2e3mZ1uYSopJD6R19fOZ0dNNf9Z=gnyksb6wg...@mail.gmail.com -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Getting sorted data from foreign server for merge join

2015-12-17 Thread Ashutosh Bapat
stead of hash join and timing reduces by approx 9%. Again the deviation in execution reduces heavily (almost by 75%). There is increase in planning time with the patch owing to firing EXPLAIN on the foreign server. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database

[HACKERS] use_remote_estimate usage for join pushdown in postgres_fdw

2015-12-11 Thread Ashutosh Bapat
Comments/suggestions are welcome. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2015-12-10 Thread Ashutosh Bapat
April 2015. It took 8 months to get that fixed. Hopefully we won't need that long to implement the hook in postgres_fdw, but that number says something about the complexity of the feature. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2015-12-10 Thread Ashutosh Bapat
On Fri, Dec 11, 2015 at 3:41 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Dec 2, 2015 at 6:45 AM, Ashutosh Bapat > <ashutosh.ba...@enterprisedb.com> wrote: > > It's been a long time since last patch on this thread was posted. I have > > started > >

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2015-12-08 Thread Ashutosh Bapat
On Thu, Dec 3, 2015 at 12:36 PM, Etsuro Fujita <fujita.ets...@lab.ntt.co.jp> wrote: > Hi Ashutosh, > > On 2015/12/02 20:45, Ashutosh Bapat wrote: > >> It's been a long time since last patch on this thread was posted. I have >> started >> to work on suppo

Re: [HACKERS] Confusing results with lateral references

2015-12-03 Thread Ashutosh Bapat
On Fri, Dec 4, 2015 at 10:58 AM, Amit Langote <langote_amit...@lab.ntt.co.jp > wrote: > On 2015/12/03 21:26, Ashutosh Bapat wrote: > > Session 1 > > postgres=# begin; > > BEGIN > > postgres=# update t1 set val = 2 where val2 = 1; > > UPDATE 1 > > >

Re: [HACKERS] Confusing results with lateral references

2015-12-03 Thread Ashutosh Bapat
| val2 | val | val2 -+--+-+-- 2 |1 | | 2 |1 | | (2 rows) It's confusing to see two rows from left join result when the table really has only a single row. Is this behaviour expected? On Thu, Dec 3, 2015 at 3:49 PM, Ashutosh Bapat < ashutosh

[HACKERS] Confusing results with lateral references

2015-12-03 Thread Ashutosh Bapat
t1, lateral (select distinct val, val2 from t2 where t2.val = t1.val) t2 for update of t1; have same semantic and should give same results. Is seeing different results expected behaviour? -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Postgres_FDW optimizations

2015-12-02 Thread Ashutosh Bapat
ey Arye > Iobeam, Inc. > > > > > -- > View this message in context: > http://postgresql.nabble.com/Postgres-FDW-optimizations-tp5875911.html > Sent from the PostgreSQL - hackers mailing list archive at Nabble.com. > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers > -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Getting sorted data from foreign server for merge join

2015-11-27 Thread Ashutosh Bapat
red by the pathkeys used for merge join. That routed through LIMIT node remains ordered. So, we actually do not need ORDER BY t1.c1 clause in the above queries. Without that clause, the tests will show difference output with and without patch. I have changed the attached patch accordingly.

Re: [HACKERS] Getting sorted data from foreign server for merge join

2015-11-20 Thread Ashutosh Bapat
On Thu, Nov 19, 2015 at 7:30 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Tue, Nov 17, 2015 at 8:33 AM, Ashutosh Bapat > <ashutosh.ba...@enterprisedb.com> wrote: > >> Although I'm usually on the side of marking things as extern whenever > >> we find it

Re: [HACKERS] Getting sorted data from foreign server for merge join

2015-11-17 Thread Ashutosh Bapat
re's no "other way" which is wrong. What's the other way you are talking about? Anway, I have updated the comment to be more readable. > Then a few lines later, you have another comment which says > the same thing again: > > +/* > + * Ignore equival

Re: [HACKERS] Getting sorted data from foreign server for merge join

2015-11-16 Thread Ashutosh Bapat
the user who declares particular collation for a foreign table var. As said upthread, different glibc implementations can cause collation ordering mismatch, this patch will be susceptible to the same problem as master/standby problem. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2015-11-09 Thread Ashutosh Bapat
5b3d5c1256de40051686f/$FILE/tdd99.pdf > > > > Sorry, may be I missed some message. but I have not received request from > you to provide feedback concerning your patch. > > See my mail on 31st August on hackers in the thread with subject "Horizontal scalability/sharding&quo

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2015-11-08 Thread Ashutosh Bapat
On Sat, Nov 7, 2015 at 12:07 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Aug 12, 2015 at 6:25 AM, Ashutosh Bapat > <ashutosh.ba...@enterprisedb.com> wrote: > > The previous patch would not compile on the latest HEAD. Here's updated > > patch. > >

[HACKERS] Getting sorted data from foreign server for merge join

2015-11-05 Thread Ashutosh Bapat
omments/suggestions are welcome. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out index 0159bf5..ddb05f2 100644 --- a/contrib/postgres_fdw/expected/pos

Re: [HACKERS] Getting sorted data from foreign server

2015-11-03 Thread Ashutosh Bapat
On Tue, Nov 3, 2015 at 11:35 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Fri, Oct 30, 2015 at 6:19 AM, Ashutosh Bapat > <ashutosh.ba...@enterprisedb.com> wrote: > > If there is a collate clause in the ORDER BY, the server crashes with > > assertion

Re: [HACKERS] Getting sorted data from foreign server

2015-10-30 Thread Ashutosh Bapat
expression since the subtree it's examining might be part of an expression which does not use the collation ultimately. On Wed, Oct 28, 2015 at 11:51 AM, Ashutosh Bapat < ashutosh.ba...@enterprisedb.com> wrote: > On Tue, Oct 27, 2015 at 6:44 PM, Fabrízio de Royes Mello < > fabriziome...@g

Re: [HACKERS] Getting sorted data from foreign server

2015-10-28 Thread Ashutosh Bapat
On Tue, Oct 27, 2015 at 6:44 PM, Fabrízio de Royes Mello < fabriziome...@gmail.com> wrote: > > > On Tue, Oct 27, 2015 at 5:26 AM, Ashutosh Bapat < > ashutosh.ba...@enterprisedb.com> wrote: > > > > > > > > On Fri, Oct 23, 2015 at 2:43

Re: [HACKERS] Getting sorted data from foreign server

2015-10-27 Thread Ashutosh Bapat
On Fri, Oct 23, 2015 at 2:43 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Oct 21, 2015 at 5:23 AM, Ashutosh Bapat > <ashutosh.ba...@enterprisedb.com> wrote: > > Increasing the sorting cost factor (when use_remote_estimates = false) > from > > 1.1 to 1

Re: [HACKERS] questions about PG update performance

2015-10-26 Thread Ashutosh Bapat
On Mon, Oct 26, 2015 at 4:09 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Mon, Oct 26, 2015 at 12:07 PM, Ashutosh Bapat < > ashutosh.ba...@enterprisedb.com> wrote: > >> >> >> On Mon, Oct 26, 2015 at 10:19 AM, Amit Kapila <amit.kapil...@gmail.com&

Re: [HACKERS] questions about PG update performance

2015-10-26 Thread Ashutosh Bapat
ack segment, and write the new value > in Redo and then do the in-place update in heap row. > > In that case, readers would pay the penalty for constructing the row. PostgreSQL will not incur the cost of reconstruction. Either writer or reader is bound to pay penalty. If the user's load is read

Re: [HACKERS] Getting sorted data from foreign server

2015-10-21 Thread Ashutosh Bapat
Here's patch with the regression fixed. On Wed, Oct 21, 2015 at 2:53 PM, Ashutosh Bapat < ashutosh.ba...@enterprisedb.com> wrote: > > > On Fri, Oct 16, 2015 at 11:33 PM, Robert Haas <robertmh...@gmail.com> > wrote: > >> On Thu, Oct 15, 2015 at 6:28 A

Re: [HACKERS] Getting sorted data from foreign server

2015-10-21 Thread Ashutosh Bapat
On Fri, Oct 16, 2015 at 11:33 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Thu, Oct 15, 2015 at 6:28 AM, Ashutosh Bapat > <ashutosh.ba...@enterprisedb.com> wrote: > > Attached is the patch which takes care of above comments. > > I spent some time on thi

Re: [HACKERS] Getting sorted data from foreign server

2015-10-15 Thread Ashutosh Bapat
On Thu, Oct 15, 2015 at 2:27 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Oct 14, 2015 at 7:30 AM, Ashutosh Bapat > <ashutosh.ba...@enterprisedb.com> wrote: > > The patch uses a factor of 1.1 (10% increase) to multiple the startup and > > total cos

Re: [HACKERS] Getting sorted data from foreign server

2015-10-14 Thread Ashutosh Bapat
PFA the patch with all the comments addressed. On Tue, Oct 13, 2015 at 10:07 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Tue, Oct 13, 2015 at 3:29 AM, Ashutosh Bapat > <ashutosh.ba...@enterprisedb.com> wrote: > >> - You consider pushing down ORDER BY

Re: [HACKERS] Getting sorted data from foreign server

2015-10-13 Thread Ashutosh Bapat
On Thu, Oct 8, 2015 at 9:39 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Tue, Oct 6, 2015 at 6:46 AM, Ashutosh Bapat > <ashutosh.ba...@enterprisedb.com> wrote: > > standard_qp_callback() sets root->query_pathkeys to pathkeys on which the > > result of que

Re: [HACKERS] Getting sorted data from foreign server

2015-10-13 Thread Ashutosh Bapat
yway those paths were getting eliminated by add_path(). I will take care of this one in the next version of patch. > -- > Jeevan B Chalke > Principal Software Engineer, Product Development > EnterpriseDB Corporation > The Enterprise PostgreSQL Company > > -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

[HACKERS] Getting sorted data from foreign server

2015-10-06 Thread Ashutosh Bapat
ign scans. 3. In all the cases, the planning time increases owing to EXPLAIN queries fired on the foreign server. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c index 697de60..25d8

Re: [HACKERS] Dependency between bgw_notify_pid and bgw_flags

2015-09-09 Thread Ashutosh Bapat
On Wed, Sep 2, 2015 at 2:02 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Mon, Aug 31, 2015 at 8:01 AM, Ashutosh Bapat > <ashutosh.ba...@enterprisedb.com> wrote: > >> Thanks. It needs testing though to see if it really works as > >> intended. Can you l

Re: [HACKERS] Horizontal scalability/sharding

2015-09-06 Thread Ashutosh Bapat
FDW writers are facing problems while implementing those hooks. In either case that should be reported on hackers. > > The PostgreSQL wiki lists 85 foreign data wrappers, and only 18 of these > have support for joins: > https://wiki.postgresql.org/wiki/Foreign_data_wrappers > > Best, > Ozgun > -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Horizontal scalability/sharding

2015-09-02 Thread Ashutosh Bapat
aiming for write scalability. > > -- > Josh Berkus > PostgreSQL Experts Inc. > http://pgexperts.com > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers > -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Proposal of Table Partition

2015-08-31 Thread Ashutosh Bapat
xpand the partitioned > table scan node into a list of partitions according to the filter and > conditions. and it can expand partitioned table JOIN node into a list of > partitions JOIN node wisely. > > For pg_dump backup action, we should dump the partition catalog, and > relpar

Re: [HACKERS] Dependency between bgw_notify_pid and bgw_flags

2015-08-31 Thread Ashutosh Bapat
On Sat, Aug 8, 2015 at 7:46 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Aug 5, 2015 at 3:33 AM, Ashutosh Bapat > <ashutosh.ba...@enterprisedb.com> wrote: > > This idea looks good. > > Thanks. It needs testing though to see if it really work

<    3   4   5   6   7   8   9   10   >