Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-03-01 Thread Etsuro Fujita
stem columns of joined relations in RETURNING, other than ctid. I'll fix that ASAP. Best regards, Etsuro Fujita *** a/contrib/postgres_fdw/deparse.c --- b/contrib/postgres_fdw/deparse.c *** *** 130,135 static void deparseReturningList(StringInfo buf, PlannerInf

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-03-07 Thread Etsuro Fujita
bulk modification directly on the remote server". It shouldn't actually begin the execution phase. + End the table update and release resources. It is normally not important And I think this one should say "Clean up following a bulk modification on the remote server".

Re: [HACKERS] Minor documentation tweak to GetForeignPlan documentation

2016-03-09 Thread Etsuro Fujita
On 2016/03/09 4:36, Robert Haas wrote: On Fri, Jan 15, 2016 at 6:20 AM, Etsuro Fujita wrote: Attached patch makes minor modification to the GetForeignPlan documentation. This adds the description about outer_plan, the new parameter added in 9.5. Good catch. Committed and back-patched to

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-03-09 Thread Etsuro Fujita
On 2016/03/08 2:35, Robert Haas wrote: On Mon, Mar 7, 2016 at 7:53 AM, Etsuro Fujita wrote: Another option to avoid such a hazard would be to remove the two changes from ExecInitModifyTable and create ExecAuxRowMarks and junk filters even in the pushdown case. I made the changes because we

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-03-10 Thread Etsuro Fujita
, you now have new ones. Personally, I think we should just commit the darned thing and you can rewrite it later if you want. If you want to rewrite it now instead, I can live with that, too. But let's figure out how we're going to move this forward. I agree with Robert. Best

Re: [HACKERS] Re: [COMMITTERS] pgsql: Only try to push down foreign joins if the user mapping OIDs mat

2016-03-13 Thread Etsuro Fujita
tempt join pushdown unless it's the same for all relevant relations. Per a suggestion from Tom Lane. Shigeru Hanada and Ashutosh Bapat, reviewed by Etsuro Fujita and KaiGai Kohei, with some further changes by me. I noticed that this breaks some citus regression tests in a minor manner. Na

Re: [HACKERS] Re: [COMMITTERS] pgsql: Only try to push down foreign joins if the user mapping OIDs mat

2016-03-13 Thread Etsuro Fujita
On 2016/03/14 11:51, Tom Lane wrote: > Etsuro Fujita writes: >> On 2016/03/13 4:46, Andres Freund wrote: >>> ... The difference apears to be the >>> check that's now in build_simple_rel() - there was nothing hitting the >>> user mapping code before for fil

[HACKERS] Obsolete comment in postgres_fdw.c

2016-03-13 Thread Etsuro Fujita
to update some related comments in src/include/nodes/relation.h and src/backend/optimizer/path/joinpath.c. Attached is a patch for that. Best regards, Etsuro Fujita *** a/contrib/postgres_fdw/postgres_fdw.c --- b/contrib/postgres_fdw/postgres_fdw.c *** ***

Re: [HACKERS] Use %u to print user mapping's umid and userid

2016-03-14 Thread Etsuro Fujita
ch for that. Best regards, Etsuro Fujita *** a/contrib/postgres_fdw/postgres_fdw.c --- b/contrib/postgres_fdw/postgres_fdw.c *** *** 65,72 enum FdwScanPrivateIndex FdwScanPrivateRetrievedAttrs, /* Integer representing the desired fetch_size */ FdwScanPrivateFetchSize, - /* Oid

Re: [HACKERS] Obsolete comment in postgres_fdw.c

2016-03-14 Thread Etsuro Fujita
On 2016/03/14 16:42, Ashutosh Bapat wrote: On Mon, Mar 14, 2016 at 9:05 AM, Etsuro Fujita mailto:fujita.ets...@lab.ntt.co.jp>> wrote: Here is the comments for foreign_join_ok in postgres_fdw.c: /* * Assess whether the join between inner and outer relations can be

[HACKERS] Missing conversion error handling in postgres_fdw

2016-03-15 Thread Etsuro Fujita
DatumGetPointer(datum); } } I think errpos.cur_attno should be set for the ctid case as well. Attached is a patch for that. Best regards, Etsuro Fujita *** a/contrib/postgres_fdw/postgres_fdw.c --- b/contrib/postgres_fdw/postgres_fdw.c *** *** 3755,3772 make

Re: [HACKERS] Missing conversion error handling in postgres_fdw

2016-03-15 Thread Etsuro Fujita
On 2016/03/16 5:55, Robert Haas wrote: On Tue, Mar 15, 2016 at 4:06 AM, Etsuro Fujita wrote: Attached is a patch for that. Hmm, I'd say you are right. Committed. Thank you for taking care of this! Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-ha

[HACKERS] Odd oid-system-column handling in postgres_fdw

2016-03-16 Thread Etsuro Fujita
when requested. I'm attaching a proposed patch for that. Best regards, Etsuro Fujita *** a/contrib/postgres_fdw/deparse.c --- b/contrib/postgres_fdw/deparse.c *** *** 287,298 foreign_expr_walker(Node *node, /* Var belongs to foreign table */ /* ! * System col

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-03-18 Thread Etsuro Fujita
On 2016/03/17 22:15, Ashutosh Bapat wrote: On Thu, Mar 17, 2016 at 4:30 PM, Etsuro Fujita mailto:fujita.ets...@lab.ntt.co.jp>> wrote: BUT: we don't make any effort to ensure that local and remote values match, so system columns other than ctid and oid should not be retriev

[HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-03-19 Thread Etsuro Fujita
retrieve any such columns in practice. Attached is a proposed patch for that. Best regards, Etsuro Fujita *** a/contrib/postgres_fdw/deparse.c --- b/contrib/postgres_fdw/deparse.c *** *** 123,129 static void deparseTargetList(StringInfo buf, Bitmapset *attrs_used,

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-03-19 Thread Etsuro Fujita
mbering the tests in postgresPlanDirectModify. That just becomes a nuisance to keep up to date as things change. Agreed. I updated the patch to address these comments. Attached is the updated version of the patch. Best regards, Etsuro Fujita *** a/contrib/postgres_fdw/deparse.c --- b/contrib/po

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-03-21 Thread Etsuro Fujita
On 2016/03/19 3:30, Robert Haas wrote: On Fri, Mar 18, 2016 at 5:15 AM, Etsuro Fujita wrote: Attached is the updated version of the patch. Committed. Thank you. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-03-21 Thread Etsuro Fujita
On 2016/03/19 4:51, Robert Haas wrote: On Thu, Mar 17, 2016 at 7:00 AM, Etsuro Fujita wrote: So, I'd like to propose: (1) when tableoids are requested from the remote server, postgres_fdw sets valid values for them locally, instead (core should support that?) Sure. and (2) when a

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-03-22 Thread Etsuro Fujita
On 2016/03/22 14:54, Ashutosh Bapat wrote: On Tue, Mar 22, 2016 at 8:03 AM, Etsuro Fujita mailto:fujita.ets...@lab.ntt.co.jp>> wrote: OK, I'll modify the patch so that the join is pushed down even if any of xmins, xmaxs, cmins, and cmaxs are requested. Do you think which

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-03-22 Thread Etsuro Fujita
On 2016/03/22 21:10, Ashutosh Bapat wrote: On Tue, Mar 22, 2016 at 5:05 PM, Etsuro Fujita mailto:fujita.ets...@lab.ntt.co.jp>> wrote: On 2016/03/22 14:54, Ashutosh Bapat wrote: Earlier in this mail chain, I suggested that the core should take care of stori

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-03-23 Thread Etsuro Fujita
lso like to propose another function that would fill system columns using these Lists when creating a scan tuple. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-03-23 Thread Etsuro Fujita
chronous call and loop on PQgetResult with CHECK_FOR_INTERRUPTS() in it. Will fix. Thanks for the report, Thom! Thanks for the advice, Michael! Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.post

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-03-25 Thread Etsuro Fujita
foreign join, proposed in [1], which has the same issue as for handling system columns in a RETURNING clause in such pushed-down UPDATE/DELETE. So I'd like to propose that approach as a common functionality.) Sorry for bringing this solution late to the table. No problem. Best regards,

Re: [HACKERS] Postgres_fdw join pushdown - INNER - FULL OUTER join combination generating wrong result

2016-03-28 Thread Etsuro Fujita
break; default: /* Should not happen, we have just check this above */ elog(ERROR, "unsupported join type %d", jointype); } wrongly pulls up remote_conds from joining relations in the FULL JOIN case. I think we should not pull up such conditions

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-03-28 Thread Etsuro Fujita
On 2016/03/25 17:16, Etsuro Fujita wrote: The approach that we discussed would minimize the code for the FDW author to write, by providing the support functions you proposed. I'll post a patch for that early next week. I added two helper functions: GetFdwScanTupleExtraDat

Re: Odd behavior in foreign table modification (Was: Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW)

2016-01-12 Thread Etsuro Fujita
On 2016/01/12 20:36, Thom Brown wrote: On 8 January 2016 at 05:08, Etsuro Fujita wrote: On 2016/01/06 20:37, Thom Brown wrote: I've run into an issue: *# UPDATE master_customers SET id = 22 WHERE id = 16 RETURNING tableoid::regclass; ERROR: CONTEXT: Remote SQL command: U

Re: [HACKERS] FDW join pushdown and scanclauses

2016-01-13 Thread Etsuro Fujita
in case a system column appears in the joinclauses it will not be considered. IIUC, we assume that such system columns are assumed to be contained in fdw_scan_tlist in the joinrel case. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make c

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-01-14 Thread Etsuro Fujita
On 2016/01/12 20:31, Rushabh Lathia wrote: On Thu, Jan 7, 2016 at 6:15 PM, Etsuro Fujita mailto:fujita.ets...@lab.ntt.co.jp>> wrote: On 2016/01/06 18:58, Rushabh Lathia wrote: .) What the need of following change ? @@ -833,9 +833,6 @@ appendWhereClause(StringIn

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-01-14 Thread Etsuro Fujita
On 2016/01/14 21:36, Rushabh Lathia wrote: On Thu, Jan 14, 2016 at 2:00 PM, Etsuro Fujita mailto:fujita.ets...@lab.ntt.co.jp>> wrote: On 2016/01/12 20:31, Rushabh Lathia wrote: On Thu, Jan 7, 2016 at 6:15 PM, Etsuro Fujita mailto:fujita.ets...@lab.ntt

Re: [HACKERS] Minor code improvements to create_foreignscan_plan/ExecInitForeignScan

2016-01-15 Thread Etsuro Fujita
On 2016/01/12 18:00, Etsuro Fujita wrote: On 2016/01/12 2:36, Alvaro Herrera wrote: I wonder, --- 2166,2213 } /* ! * If rel is a base relation, detect whether any system columns are ! * requested from the rel. (If rel is a join relation, rel->relid will

Re: [HACKERS] Minor comment update in setrefs.c

2016-01-15 Thread Etsuro Fujita
On 2015/12/11 2:21, Robert Haas wrote: On Tue, Dec 8, 2015 at 6:16 AM, Etsuro Fujita wrote: Attached is a small patch to adjust a comment in setrefs.c; in set_foreignscan_references, fdw_recheck_quals also gets adjusted to reference foreign scan tuple, in case of a foreign join, so I added

[HACKERS] Minor documentation tweak to GetForeignPlan documentation

2016-01-15 Thread Etsuro Fujita
Attached patch makes minor modification to the GetForeignPlan documentation. This adds the description about outer_plan, the new parameter added in 9.5. Best regards, Etsuro Fujita *** a/doc/src/sgml/fdwhandler.sgml --- b/doc/src/sgml/fdwhandler.sgml *** *** 178,184

Re: [HACKERS] Minor code improvements to create_foreignscan_plan/ExecInitForeignScan

2016-01-18 Thread Etsuro Fujita
On 2016/01/15 19:00, Etsuro Fujita wrote: On 2016/01/12 18:00, Etsuro Fujita wrote: On 2016/01/12 2:36, Alvaro Herrera wrote: I wonder, --- 2166,2213 } /* ! * If rel is a base relation, detect whether any system columns are ! * requested from the rel. (If rel

Re: Odd behavior in foreign table modification (Was: Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW)

2016-01-18 Thread Etsuro Fujita
On 2016/01/08 14:08, Etsuro Fujita wrote: On 2016/01/07 21:50, Etsuro Fujita wrote: On 2016/01/06 20:37, Thom Brown wrote: I've run into an issue: *# UPDATE master_customers SET id = 22 WHERE id = 16 RETURNING tableoid::regclass; ERROR: CONTEXT: Remote SQL command: UPDATE public.cust

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-01-18 Thread Etsuro Fujita
#3, #1, #2). Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: Odd behavior in foreign table modification (Was: Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW)

2016-01-20 Thread Etsuro Fujita
On 2016/01/20 3:42, Robert Haas wrote: On Tue, Jan 19, 2016 at 1:59 AM, Etsuro Fujita wrote: I've run into an issue: *# UPDATE master_customers SET id = 22 WHERE id = 16 RETURNING tableoid::regclass; ERROR: CONTEXT: Remote SQL command: UPDATE public.customers SET id = 22 WHERE ((id

Re: Odd behavior in foreign table modification (Was: Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW)

2016-01-21 Thread Etsuro Fujita
On 2016/01/21 5:06, Robert Haas wrote: On Wed, Jan 20, 2016 at 4:50 AM, Etsuro Fujita wrote: My concern about that is that would make the code in deparseTargetList() complicated. Essentially, I think your propossal needs a two-pass algorithm for deparseTargetList; (1) create an integer List

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-01-21 Thread Etsuro Fujita
se docs, including a fix for a wrong copy-and-paste. Best regards, Etsuro Fujita [1] http://www.postgresql.org/message-id/ca+tgmoz40j2uc5ac1nxu03oj4crvolks15xx+ptfp-1u-8z...@mail.gmail.com *** a/contrib/postgres_fdw/deparse.c --- b/contrib/postgres_fdw/deparse.c *** *** 823,829 **

Re: Odd behavior in foreign table modification (Was: Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW)

2016-01-21 Thread Etsuro Fujita
On 2016/01/19 19:04, Thom Brown wrote: On 12 January 2016 at 11:49, Etsuro Fujita wrote: On 2016/01/12 20:36, Thom Brown wrote: On 8 January 2016 at 05:08, Etsuro Fujita wrote: On 2016/01/06 20:37, Thom Brown wrote: I've run into an issue: *# UPDATE master_customers SET id = 22

Re: [HACKERS] Minor code improvements to create_foreignscan_plan/ExecInitForeignScan

2016-01-21 Thread Etsuro Fujita
On 2016/01/21 7:04, Alvaro Herrera wrote: Etsuro Fujita wrote: On second thought, I noticed that detecting whether we see a system column that way needs more cycles in cases where the reltargetlist and the restriction clauses don't contain any system columns. ISTM that such cases are r

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-01-26 Thread Etsuro Fujita
ETE, not just UPDATE. (IsForeignRelUpdatable discussed above reports not only the updatability but the insertability and deletability of a foreign table!). So, +1 for leaving that as-is. Apart from this perform sanity testing on the new patch and things working as expected. Thanks for the rev

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-01-26 Thread Etsuro Fujita
On 2016/01/26 22:57, Rushabh Lathia wrote: On Tue, Jan 26, 2016 at 4:15 PM, Etsuro Fujita mailto:fujita.ets...@lab.ntt.co.jp>> wrote: On 2016/01/25 17:03, Rushabh Lathia wrote: int IsForeignRelUpdatable (Relation rel); Documentation for IsForeignUpd

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-01-27 Thread Etsuro Fujita
On 2016/01/27 12:20, Etsuro Fujita wrote: On 2016/01/26 22:57, Rushabh Lathia wrote: On Tue, Jan 26, 2016 at 4:15 PM, Etsuro Fujita mailto:fujita.ets...@lab.ntt.co.jp>> wrote: On 2016/01/25 17:03, Rushabh Lathia wrote: int IsForeignRelUpdatable (Relati

[HACKERS] Minor improvement to fdwhandler.sgml

2016-01-27 Thread Etsuro Fujita
Here is a small patch to do s/for/For/ to two section titles in fdwhandlers.sgml, for consistency. Best regards, Etsuro Fujita diff --git a/doc/src/sgml/fdwhandler.sgml b/doc/src/sgml/fdwhandler.sgml index dc2d890..9c8406c 100644 --- a/doc/src/sgml/fdwhandler.sgml +++ b/doc/src/sgml

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-01-27 Thread Etsuro Fujita
Am I right? Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-01-27 Thread Etsuro Fujita
On 2016/01/28 15:20, Rushabh Lathia wrote: On Thu, Jan 28, 2016 at 11:33 AM, Etsuro Fujita mailto:fujita.ets...@lab.ntt.co.jp>> wrote: On 2016/01/27 21:23, Rushabh Lathia wrote: If I understood correctly, above documentation means, that if FDW have DMLPushdow

Re: [HACKERS] Minor code improvements to create_foreignscan_plan/ExecInitForeignScan

2016-01-27 Thread Etsuro Fujita
On 2016/01/28 12:13, Robert Haas wrote: On Thu, Jan 21, 2016 at 5:55 AM, Etsuro Fujita wrote: On 2016/01/21 7:04, Alvaro Herrera wrote: Etsuro Fujita wrote: On second thought, I noticed that detecting whether we see a system column that way needs more cycles in cases where the reltargetlist

Re: Odd behavior in foreign table modification (Was: Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW)

2016-01-28 Thread Etsuro Fujita
On 2016/01/28 12:58, Robert Haas wrote: On Thu, Jan 21, 2016 at 4:05 AM, Etsuro Fujita wrote: By the way, I'm not too sure I understand the need for the core changes that are part of this patch, and I think that point merits some discussion. Whenever you change core like this, you'r

Re: [HACKERS] Minor code improvements to create_foreignscan_plan/ExecInitForeignScan

2016-01-28 Thread Etsuro Fujita
On 2016/01/28 18:15, Alvaro Herrera wrote: Etsuro Fujita wrote: On 2016/01/28 12:13, Robert Haas wrote: I don't think this is a good idea. Most of the time, no system columns will be present, and we'll just be scanning the Bitmapset twice rather than once. Sure, that doesn&#

Re: [HACKERS] New committer

2016-01-28 Thread Etsuro Fujita
deserved. Congratulations, Dean! Congratulations! Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-01-29 Thread Etsuro Fujita
e parameterized-foreign-join-path cases, though that would work well for the unparameterized-foreign-join-path cases. We don't support parameterized-foreign-join paths for 9.6? Best regards, Etsuro Fujita [1] http://www.postgresql.org/message-id/5666b59f.6010...@lab.ntt.co.jp -- Se

[HACKERS] Copy-pasto in the ExecForeignDelete documentation

2016-02-01 Thread Etsuro Fujita
columns, but the I don't think the data is referenced by the AFTER ROW DELETE triggers. Attached is a patch to fix that. The patch also avoids adding an unnecessary RETURNING clause to DELETE when deparsing a remote DELETE statement in postgres_fdw. I'll add this to the next CF. Best regar

Re: [HACKERS] Minor code improvements to create_foreignscan_plan/ExecInitForeignScan

2016-02-02 Thread Etsuro Fujita
On 2016/02/03 3:31, Alvaro Herrera wrote: Etsuro Fujita wrote: Done. Attached is an updated version of the patch. Pushed, thanks. Thank you! I kinda wonder why this struct member has a name that doesn't match the naming convention in the rest of the struct, and also why isn

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-02-03 Thread Etsuro Fujita
On 2016/01/28 15:20, Rushabh Lathia wrote: On Thu, Jan 28, 2016 at 11:33 AM, Etsuro Fujita mailto:fujita.ets...@lab.ntt.co.jp>> wrote: On 2016/01/27 21:23, Rushabh Lathia wrote: If I understood correctly, above documentation means, that if FDW have DMLPushdow

Re: [HACKERS] 2016-01 Commitfest

2016-02-03 Thread Etsuro Fujita
On 2016/02/04 12:04, Robert Haas wrote: On Wed, Feb 3, 2016 at 10:00 PM, Noah Misch wrote: Thank you. +1. Thank you! Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] Copy-pasto in the ExecForeignDelete documentation

2016-02-03 Thread Etsuro Fujita
On 2016/02/04 0:13, Robert Haas wrote: On Mon, Feb 1, 2016 at 5:26 AM, Etsuro Fujita wrote: I don't think the data is referenced by the AFTER ROW DELETE triggers. Why do you think that? And why would DELETE triggers be different from UPDATE triggers, which do something similar? A

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-04 Thread Etsuro Fujita
ill be used by custom joins, so I think it would be better to put that function somewhere in the /optimizer directory (pathnode.c?). Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-04 Thread Etsuro Fujita
On 2016/01/29 17:52, Ashutosh Bapat wrote: On Fri, Jan 29, 2016 at 2:05 PM, Etsuro Fujita mailto:fujita.ets...@lab.ntt.co.jp>> wrote: On 2016/01/29 1:26, Ashutosh Bapat wrote: Here is the summary of changes from the last set of patches 2. Included fix for EvalPl

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-04 Thread Etsuro Fujita
On 2016/02/04 17:58, Etsuro Fujita wrote: On 2016/02/04 8:00, Robert Haas wrote: On Wed, Feb 3, 2016 at 5:56 PM, Robert Haas wrote: On Wed, Feb 3, 2016 at 12:08 PM, Ashutosh Bapat wrote: PFA patches with naming conventions similar to previous ones. pg_fdw_core_v7.patch: core changes

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-02-04 Thread Etsuro Fujita
of the patch. You don't need to apply the patch fdw-foreign-modify-rmh-v2.patch attached before. Thanks for the review! Best regards, Etsuro Fujita *** a/contrib/postgres_fdw/deparse.c --- b/contrib/postgres_fdw/deparse.c *** *** 1069,1074 deparseUpdateSql(StringInf

Re: Odd behavior in foreign table modification (Was: Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW)

2016-02-04 Thread Etsuro Fujita
On 2016/02/05 12:28, Robert Haas wrote: On Thu, Jan 28, 2016 at 7:36 AM, Etsuro Fujita wrote: Attached is that version of the patch. I think that postgres_fdw might be able to insert a tableoid value in the returned slot in e.g., postgresExecForeignInsert if AFTER ROW Triggers or RETURNING

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-05 Thread Etsuro Fujita
, IIUC, I think the patch fails to adjust the targetlist of the top plan created that way, to output the fdw_scan_tlist, as discussed in [1] (ie, I think the attached patch is needed, which is created on top of your patch pg_fdw_join_v8.patch). Best regards, Etsuro Fujita [1] http://w

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-05 Thread Etsuro Fujita
to be executed, thus even if root->rowMarks is NIL, EPQ check can fire and we will need alternate local plan. Yeah, I think that is true, but if root->rowMarks==NIL, we won't have non-target foreign tables, and therefore postgresGetForeignJoinPaths() will never be called. No? Best

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-08 Thread Etsuro Fujita
om the local table tab, the EvalPlanQual recheck executed should succeed.) The reason for that is that the targetlist of the local join plan is the same as for the ForeignScan, which outputs neither foo.a nor bar.a required as an argument of the function add(). Best regards, Etsuro Fujit

[HACKERS] Use %u to print user mapping's umid and userid

2016-02-08 Thread Etsuro Fujita
Here is a patch to use %u not %d to print umid and userid. Best regards, Etsuro Fujita *** a/contrib/postgres_fdw/connection.c --- b/contrib/postgres_fdw/connection.c *** *** 159,165 GetConnection(UserMapping *user, bool will_prep_stmt) entry->have_error = false; en

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-02-12 Thread Etsuro Fujita
s. Also: "When the query doesn't has the clause, the FDW must also increment the row count for the ForeignScanState node in the EXPLAIN ANALYZE case." Should read "doesn't have" Will fix. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pg

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-02-12 Thread Etsuro Fujita
on that both postgresBeginForeignScan and postgresBeginForeignModify can call. execute_dml_stmt() has some of the same disease. Will do. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-02-14 Thread Etsuro Fujita
On 2016/02/12 21:46, Robert Haas wrote: On Fri, Feb 12, 2016 at 7:19 AM, Etsuro Fujita wrote: I think that displaying target lists would be confusing for users. Here is an example: EXPLAIN (verbose, costs off) DELETE FROM rem1; -- can be pushed down QUERY

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-14 Thread Etsuro Fujita
az'); CREATE FOREIGN TABLE postgres=# select * from foo, bar, baz where foo.a = bar.a and bar.a = baz.a and foo.a < 10 and bar.a < 10 and baz.a < 10; Attached is a patch to avoid the breakage. Best regards, Etsuro Fujita *** a/contrib/postgres_fdw/postgres_fdw.c --- b/contrib/po

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-02-15 Thread Etsuro Fujita
On 2016/02/15 15:20, Rushabh Lathia wrote: On Fri, Feb 12, 2016 at 5:40 PM, Etsuro Fujita mailto:fujita.ets...@lab.ntt.co.jp>> wrote: As a result of our discussions, we reached a conclusion that the DML pushdown APIs should be provided together with existing APIs s

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-15 Thread Etsuro Fujita
any higher level joins. Maybe I'm missing something, but I don't understand why such a change in those lists happens. Could you explain about that in more detail? Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-15 Thread Etsuro Fujita
, but I don't think that that needs to be considered because we have this at the beginning of postgresGetGForeignJoinPaths: /* * Skip if this join combination has been considered already. */ if (joinrel->fdw_private) return; Best regards, Etsuro Fujita -- Sent v

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-15 Thread Etsuro Fujita
On 2016/02/16 16:02, Ashutosh Bapat wrote: On Tue, Feb 16, 2016 at 12:26 PM, Etsuro Fujita mailto:fujita.ets...@lab.ntt.co.jp>> wrote: On 2016/02/16 15:22, Ashutosh Bapat wrote: During join planning, the planner tries multiple combinations of joining rel

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-02-18 Thread Etsuro Fujita
On 2016/02/12 21:19, Etsuro Fujita wrote: Comments on specific points follow. This seems to need minor rebasing in the wake of the join pushdown patch. Will do. Done. + /* Likewise for ForeignScan that has pushed down INSERT/UPDATE/DELETE */ + if (IsA(plan, ForeignScan

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-18 Thread Etsuro Fujita
On 2016/02/16 16:40, Etsuro Fujita wrote: On 2016/02/16 16:02, Ashutosh Bapat wrote: On Tue, Feb 16, 2016 at 12:26 PM, Etsuro Fujita mailto:fujita.ets...@lab.ntt.co.jp>> wrote: On 2016/02/16 15:22, Ashutosh Bapat wrote: During join planning, the planner tries mu

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-02-22 Thread Etsuro Fujita
On 2016/02/22 20:13, Rushabh Lathia wrote: I did another round of review for the latest patch and well as performed the sanity test, and haven't found any functional issues. Found couple of issue, see in-line comments for the same. Thanks! On Thu, Feb 18, 2016 at 3:15 PM, Etsuro F

[HACKERS] Push down more full joins in postgres_fdw

2016-08-19 Thread Etsuro Fujita
."T \ 1") ss1(c1, c2, c3, c4) INNER JOIN (SELECT ROW("C 1", c2, c3, c4, c5, c6, c7, c8), "C 1" FROM "S 1"."T 1") ss2(c1, c2) ON (TRUE)) WHERE ((ss1.c3 = ss2.\ c2)) ORDER BY ss1.c4 ASC NULLS LAST, ss1.c3 ASC NULLS LAST (6 rows) I'll add this

Re: [HACKERS] Oddity in EXPLAIN for foreign/custom join pushdown plans

2016-08-23 Thread Etsuro Fujita
On 2016/08/10 5:19, Robert Haas wrote: On Mon, Aug 8, 2016 at 12:22 AM, Etsuro Fujita wrote: One thing we need to do to leave that as is would be to fix a bug that I pointed out upthred. Let me explain about that again. The EXPLAIN command selects relation aliases to be used in printing a

Collective typos in contrib/postgres_fdw (Was: Re: [HACKERS] Incorrect comment in contrib/postgres_fdw/deparse.c)

2016-08-23 Thread Etsuro Fujita
On 2016/04/04 20:11, Etsuro Fujita wrote: I found an incorrect comment in contrib/postgres_fdw/deparse.c: s/FOR SELECT or FOR SHARE/FOR UPDATE or FOR SHARE/ Attached is a patch to fix that comment. Other than this, I ran into some typos in contrib/postgres_fdw, while working on join

Re: [HACKERS] postgres_fdw : altering foreign table not invalidating prepare statement execution plan.

2016-08-24 Thread Etsuro Fujita
I'm missing something, though. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Oddity in EXPLAIN for foreign/custom join pushdown plans

2016-08-24 Thread Etsuro Fujita
On 2016/08/25 1:08, Robert Haas wrote: On Tue, Aug 23, 2016 at 11:18 PM, Etsuro Fujita wrote: OK, I think we should fix the issue that postgres_fdw produces incorrect aliases for joining relations shown in the Relations line in EXPLAIN for a join pushdown query like the above) in advance of

Re: Collective typos in contrib/postgres_fdw (Was: Re: [HACKERS] Incorrect comment in contrib/postgres_fdw/deparse.c)

2016-08-24 Thread Etsuro Fujita
On 2016/08/25 5:29, Robert Haas wrote: On Wed, Aug 24, 2016 at 2:41 AM, Etsuro Fujita wrote: On 2016/04/04 20:11, Etsuro Fujita wrote: I found an incorrect comment in contrib/postgres_fdw/deparse.c: s/FOR SELECT or FOR SHARE/FOR UPDATE or FOR SHARE/ Attached is a patch to fix that comment

Re: [HACKERS] Odd oid-system-column handling in postgres_fdw

2016-08-29 Thread Etsuro Fujita
On 2016/08/26 22:35, Heikki Linnakangas wrote: On 04/05/2016 11:15 AM, Etsuro Fujita wrote: On 2016/03/16 16:25, Etsuro Fujita wrote: PG9.5 allows us to add an oid system column to foreign tables, using ALTER FOREIGN TABLE SET WITH OIDS, but currently, that column reads as zeroes in

Re: [HACKERS] Confusing docs about GetForeignUpperPaths in fdwhandler.sgml

2016-08-31 Thread Etsuro Fujita
On 2016/08/01 21:14, Etsuro Fujita wrote: I noticed that the following note about direct modification via GetForeignUpperPaths in fdwhandler.sgml is a bit confusing. We have another approach using PlanDirectModify, so that should be reflected in the note as well. Please find attached a patch

Re: [HACKERS] Push down more full joins in postgres_fdw

2016-09-02 Thread Etsuro Fujita
ttached is an updated version of the patch. Other changes: * Add a bit more regression test * Revise code/comments * Cleanups Thanks for the comments! Best regards, Etsuro Fujita postgres-fdw-more-full-join-pushdown-v2.patch Description: binary/octet-stream -- Sent via pgsql-hacker

Re: [HACKERS] Confusing docs about GetForeignUpperPaths in fdwhandler.sgml

2016-09-02 Thread Etsuro Fujita
Hi Robert, Thanks for the comments! On 2016/09/02 11:55, Robert Haas wrote: On Mon, Aug 1, 2016 at 5:44 PM, Etsuro Fujita wrote: I noticed that the following note about direct modification via GetForeignUpperPaths in fdwhandler.sgml is a bit confusing. We have another approach using

[HACKERS] Push down more UPDATEs/DELETEs in postgres_fdw

2016-09-06 Thread Etsuro Fujita
patch [1]. So, for testing, please apply the patch in [1] first. I'll add this to the the November commitfest. Best regards, Etsuro Fujita [1] https://www.postgresql.org/message-id/165b-5fb1-8bfa-b1b8-c2758dbe0...@lab.ntt.co.jp postgres-fdw-more-update-pushdown-WIP.patch Descr

Re: [HACKERS] Push down more full joins in postgres_fdw

2016-09-08 Thread Etsuro Fujita
On 2016/09/06 22:07, Ashutosh Bapat wrote: On Fri, Sep 2, 2016 at 3:55 PM, Etsuro Fujita mailto:fujita.ets...@lab.ntt.co.jp>> wrote: On 2016/08/22 15:49, Ashutosh Bapat wrote: 1. deparsePlaceHolderVar looks odd - each of the deparse* function is named as d

Re: [HACKERS] Push down more UPDATEs/DELETEs in postgres_fdw

2016-09-08 Thread Etsuro Fujita
But it's not needed if we are pushing down the query. If we eliminate the targetlist of the query, we could construct a remote query without having subquery in it, making it more readable. Will try to do so also. Thanks for the comments! Best regards, Etsuro Fujita -- Sent via pgsql

Re: [HACKERS] Push down more full joins in postgres_fdw

2016-09-09 Thread Etsuro Fujita
On 2016/09/08 19:51, Etsuro Fujita wrote: On 2016/09/06 22:07, Ashutosh Bapat wrote: This patch tries to do two things at a time 1. support join pushdown for FULL join when the joining relations have remote conditions 2. better support for fetching placeholder vars, whole row references and

Re: [HACKERS] Push down more full joins in postgres_fdw

2016-09-12 Thread Etsuro Fujita
On 2016/09/09 21:35, Etsuro Fujita wrote: On 2016/09/08 19:51, Etsuro Fujita wrote: On 2016/09/06 22:07, Ashutosh Bapat wrote: This patch tries to do two things at a time 1. support join pushdown for FULL join when the joining relations have remote conditions 2. better support for fetching

Re: [HACKERS] Push down more full joins in postgres_fdw

2016-09-26 Thread Etsuro Fujita
Will fix. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Push down more full joins in postgres_fdw

2016-09-26 Thread Etsuro Fujita
->parse->rtable)), instead? Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-09-26 Thread Etsuro Fujita
On 2016/09/21 0:40, Robert Haas wrote: On Fri, Jul 1, 2016 at 3:10 AM, Etsuro Fujita wrote: On 2016/04/14 4:57, Robert Haas wrote: 1. For a regular FDW scan, zero the xmin, xmax, and cid of the tuple before returning it from postgres_fdw, so that we don't expose the datum-tuple fields

Re: [HACKERS] Push down more full joins in postgres_fdw

2016-09-26 Thread Etsuro Fujita
On 2016/09/26 18:06, Ashutosh Bapat wrote: On Mon, Sep 26, 2016 at 1:05 PM, Etsuro Fujita wrote: ISTM that the use of the same RTI for subqueries in multi-levels in a remote SQL makes the SQL a bit difficult to read. How about using the position of the join rel in join_rel_list, (more

Re: [HACKERS] Push down more full joins in postgres_fdw

2016-09-26 Thread Etsuro Fujita
On 2016/09/26 20:20, Ashutosh Bapat wrote: On Mon, Sep 26, 2016 at 4:06 PM, Etsuro Fujita wrote: On 2016/09/26 18:06, Ashutosh Bapat wrote: On Mon, Sep 26, 2016 at 1:05 PM, Etsuro Fujita wrote: ISTM that the use of the same RTI for subqueries in multi-levels in a remote SQL makes the SQL

Re: [HACKERS] Push down more full joins in postgres_fdw

2016-09-28 Thread Etsuro Fujita
use that routine skips all its work after the first call for that joinrel, by checking to see if joinrel->fdw_private is not NULL. So, I think it's reasonable to get the position by list_length in that routine. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-ha

Re: [HACKERS] Push down more full joins in postgres_fdw

2016-09-28 Thread Etsuro Fujita
On 2016/09/26 16:30, Etsuro Fujita wrote: On 2016/09/13 14:17, Ashutosh Bapat wrote: It won't remain minimal as the number of paths created increases, increasing the number of times a query is deparsed. We deparse query every time time we cost a path for a relation with use_remote_esti

Re: [HACKERS] Another oddity in handling of WCO constraints in postgres_fdw

2017-10-04 Thread Etsuro Fujita
m ft where a % 4 = 0 with check option; how is the WCO constraint (ie, a % 4 = 0) ensured remotely, which is different from the constraint on the foreign table (ie, a % 2 = 0)? Maybe I'm missing something, though. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-h

Re: [HACKERS] Another oddity in handling of WCO constraints in postgres_fdw

2017-10-05 Thread Etsuro Fujita
here we ignore what's happening on remote side and enforce whatever we could enforce locally. Local server should make sure that any rows sent from local server to the remote server do not violate any local WCO. Seems odd (and too restrictive) to me too. Best regards, Etsuro Fujita

  1   2   3   4   5   6   7   8   9   10   >