Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2015-08-04 Thread Shigeru Hanada
gt; Rest of the patch looks fine. Thanks -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [idea] more aggressive join pushdown on postgres_fdw

2015-06-05 Thread Shigeru HANADA
stgres_fdw case. If this assumption is correct, we need a mapping between a local ordinary table and a foreign table which points remote replicated table. -- Shigeru HANADA shigeru.han...@gmail.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes t

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2015-05-22 Thread Shigeru Hanada
Thank for your comments. 2015-05-21 23:11 GMT+09:00 Robert Haas : > On Sat, May 16, 2015 at 9:04 AM, Shigeru Hanada > wrote: >> d) All relations must have the same effective user id >> This check is done with userid stored in PgFdwRelationInfo, which is >> valid only w

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-05-15 Thread Shigeru Hanada
(or at least via CSP I/F). -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-05-12 Thread Shigeru Hanada
of a join tree. Some FDW might hope to use sub-plan generation, but we don't have any concrete use case as of now. -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2015-05-02 Thread Shigeru HANADA
Thanks for the comments. 2015/05/01 22:35、Robert Haas のメール: > On Mon, Apr 27, 2015 at 5:07 AM, Shigeru Hanada > wrote: >> 2015-04-27 11:00 GMT+09:00 Kouhei Kaigai : >>> Hanada-san, could you adjust your postgres_fdw patch according to >>> the above new (pre

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-04-27 Thread Shigeru HANADA
Foreign Scan (cost=100.00..101.00 rows=50 width=716) Output: b.bid, b.bbalance, b.filler, t.tid, t.bid, t.tbalance, t.filler Relations: (public.pgbench_branches b) CROSS JOIN (public.pgbench_tellers t) Remote SQL: SELECT l.a1, l.a2, l.a3, r.a1, r.a2, r.a3, r.a4 FROM (SELECT l.a9, l.a10, l.a11 FROM (SELECT bid a9, bbalance a10, filler a11 FROM public.pgbench_branches) l) l (a1, a2, a3) CROSS JOIN (SELECT r.a9, r.a10, r.a11, r.a12 FROM (SELECT tid a9, bid a10, tbalance a11, filler a12 FROM public.pgbench_tellers) r) r (a1, a2, a3, a4) WHERE ((l.a1 = r.a2)) (4 rows) Thoughts? Regards, -- Shigeru HANADA shigeru.han...@gmail.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-04-24 Thread Shigeru HANADA
-- Foreign Scan (cost=100.00..101.00 rows=50 width=716) Output: b.bid, b.bbalance, b.filler, t.tid, t.bid, t.tbalance, t.fille

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-04-20 Thread Shigeru HANADA
changed to fix Vars in ps_tlist of CustomScan and ForeignScan. For this change, new BitmapSet function bms_shift_members is added. set_deparse_planstate is also changed to pass ps_tlist as namespace for deparsing. These chanes seems reasonable, so I mark this patch as “ready for committers” to hear committers' thoughts. Regards, -- Shigeru HANADA shigeru.han...@gmail.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-04-14 Thread Shigeru HANADA
elation. Oops, that’s right. Attached is the revised version. I chose fully qualified name, schema.relname [alias] for the output. It would waste some cycles during planning if that is not for EXPLAIN, but it seems difficult to get a list of name of relations in ExplainForeignScan() phase, be

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-03-25 Thread Shigeru HANADA
ze a query which accesses multiple parent tables, each is inherited by foreign tables and partitioned with identical join key, by building a path tree which joins sharded tables first, and then union those results. -- Shigeru HANADA shigeru.han...@gmail.com -- Sent via pgsql-hackers ma

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-03-25 Thread Shigeru HANADA
ld remote- > join query. Another idea is to pass “found” as parameter to FDW handler, and let FDW to decide to skip or not. Some of FDWs (and some of CSP?) might want to be conscious of join combination. — Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-03-25 Thread Shigeru HANADA
2015/03/25 18:53、Ashutosh Bapat のメール: > > > On Wed, Mar 25, 2015 at 3:14 PM, Shigeru HANADA > wrote: > > Or bottom of make_join_rel(). IMO build_join_rel() is responsible for just > building (or searching from a list) a RelOptInfo for given relids. After > th

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-03-25 Thread Shigeru HANADA
2015/03/25 19:09、Kouhei Kaigai のメール: >> On Wed, Mar 25, 2015 at 3:14 PM, Shigeru HANADA >> wrote: >> Or bottom of make_join_rel(). IMO build_join_rel() is responsible for >> just building (or searching from a list) a RelOptInfo for given relids. >> After

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-03-25 Thread Shigeru HANADA
r particular relid combination, and there SpecialJoinInfo and restrictlist (conditions specified in JOIN-ON and WHERE), so it seems promising for FDW cases. Though I’m not sure that it also fits custom join provider’s requirements. — Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-03-24 Thread Shigeru HANADA
reak > down join relations. With applying your patch, regression tests of “updatable view” failed. regression.diff contains some errors like this: ! ERROR: could not find RelOptInfo for given relids Could you check that? — Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-03-16 Thread Shigeru Hanada
level) is an idea, but it would make planning time longer (and use more cycle and memory). Tom's idea sounds good for saving the path b), but I worry that whether FDW can get enough information at that timing, just before set_cheapest. It would not be good I/F if each FDW n

Re: [HACKERS] Join push-down support for foreign tables

2015-03-05 Thread Shigeru Hanada
ssues from Kaigai-san and Ashutosu, whole-row reference handling and use of get_joinrel_parampathinfo(). 2015-03-05 22:00 GMT+09:00 Shigeru Hanada : > Hi Ashutosh, thanks for the review. > > 2015-03-04 19:17 GMT+09:00 Ashutosh Bapat : >> In create_foreignscan_path() we have lines li

Re: [HACKERS] Join push-down support for foreign tables

2015-03-05 Thread Shigeru Hanada
t; -- > Foreign Scan (cost=100.00..132.00 rows=2560 width=8) >Output: (val + val2) >Remote SQL: SELECT r.a_0, r.a_1 FROM (SELECT val, NULL FROM public.lt) l > (a_0, a_1) INNER JOIN (SELECT val, val2 FROM public.lt) r (a_0, a_1) ON (( > (r.a

Re: [HACKERS] Join push-down support for foreign tables

2015-03-04 Thread Shigeru Hanada
rs t where t.bid = b.bid and t.tid < 10; ERROR: ctid is NULL 2015-03-03 21:34 GMT+09:00 Shigeru Hanada : > I rebased "join push-down" patch onto Kaigai-san's Custom/Foreign Join > v6 patch. I posted some comments to v6 patch in this post: > > http://www.postgresql

Re: [HACKERS] Join push-down support for foreign tables

2015-03-04 Thread Shigeru Hanada
2015-03-04 17:00 GMT+09:00 Etsuro Fujita : > On 2015/03/03 21:34, Shigeru Hanada wrote: >> >> I rebased "join push-down" patch onto Kaigai-san's Custom/Foreign Join >> v6 patch. > > > Thanks for the work, Hanada-san and KaiGai-san! > > Maybe I

Re: [HACKERS] Join push-down support for foreign tables

2015-03-03 Thread Shigeru Hanada
I adjusted the condition to check scanrelid instead. >> >> Is this issue fixed by v5 custom/foreign join patch? >> > Yes, please rebase it. > > -- > NEC OSS Promotion Center / PG-Strom Project > KaiGai Kohei > -- Shigeru HANADA foreign_join_v3.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-03-03 Thread Shigeru Hanada
ehalf Of Kouhei Kaigai >> Sent: Tuesday, March 03, 2015 6:31 PM >> To: Kaigai Kouhei(海外 浩平); Robert Haas >> Cc: Tom Lane; pgsql-hackers@postgreSQL.org; Shigeru Hanada >> Subject: Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API) >> >> The attach

Re: [HACKERS] Join push-down support for foreign tables

2015-03-03 Thread Shigeru Hanada
gt; at postgres_fdw.c:3293 > #1 0x008e51d6 in errfinish (dummy=0) at elog.c:436 > #2 0x008935cd in tidin (fcinfo=0x7fffc257e8a0) at tid.c:69 > /home/kaigai/repo/sepgsql/src/backend/utils/adt/tid.c:69:1734:beg:0x8935cd > (gdb) p str > $6 = 0x1d17cf7 "foo" Join push-down underlying UPDATE or DELETE requires ctid as its output, but it seems not fully supported. I'm fixing this issue now. Regards, -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Join push-down support for foreign tables

2015-03-03 Thread Shigeru Hanada
s in a query. Do you have a good idea? I'd like to vote for keeping current aliasing style, use "l" and "r" for join source relations, and use a_0, a_1, ... for each column of them. -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Join push-down support for foreign tables

2015-03-03 Thread Shigeru Hanada
stgres_fdw. > Another issue: > > # EXPLAIN VERBOSE SELECT NULL FROM (SELECT people.id FROM people INNER JOIN > countries ON people.country_id = countries.id LIMIT 3) x; > ERROR: could not open relation with OID 0 Good catch. In my quick trial, removing LIMIT3 avoids this error. I&#x

Re: [HACKERS] Join push-down support for foreign tables

2015-03-02 Thread Shigeru Hanada
nPaths() to keep the policy that all interface between core and FDW should be in FdwRoutine, instead of using hook function. Now I'm writing document about it, and will post it in a day. 2015-02-19 16:19 GMT+09:00 Shigeru Hanada : > 2015-02-17 10:39 GMT+09:00 Kouhei Kaigai : >> Let me put

Re: [HACKERS] Join push-down support for foreign tables

2015-02-18 Thread Shigeru Hanada
> A few functions added in src/backend/foreign/foreign.c are not > called by anywhere, at this moment. > > create_plan_recurse() is reverted to static. It is needed for custom- > join enhancement, if no other infrastructure can support. I made it back to static because I tho

Re: [HACKERS] Join push-down support for foreign tables

2015-02-15 Thread Shigeru Hanada
ion? > > Thanks, > -- > NEC OSS Promotion Center / PG-Strom Project > KaiGai Kohei > > >> -Original Message- >> From: pgsql-hackers-ow...@postgresql.org >> [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Shigeru Hanada >> Sent: Monday, Februa

Re: [HACKERS] Join push-down support for foreign tables

2015-02-15 Thread Shigeru Hanada
before, N-way (not only 2-way) joins should be supported in the first version by construct SELECT SQL by containing source query in FROM clause as inline views (a.k.a. from clause subquery). 2014-12-26 13:48 GMT+09:00 Shigeru Hanada : > 2014-12-16 1:22 GMT+09:00 Robert Haas : >> On Mon, Dec

Re: [HACKERS] Join push-down support for foreign tables

2014-12-25 Thread Shigeru Hanada
2014-12-16 1:22 GMT+09:00 Robert Haas : > On Mon, Dec 15, 2014 at 3:40 AM, Shigeru Hanada > wrote: >> I'm working on $SUBJECT and would like to get comments about the >> design. Attached patch is for the design below. > > I'm glad you are working on this. &

Re: [HACKERS] Join push-down support for foreign tables

2014-12-25 Thread Shigeru Hanada
2014-12-16 0:45 GMT+09:00 Tom Lane : > Shigeru Hanada writes: >> I'm working on $SUBJECT and would like to get comments about the >> design. Attached patch is for the design below. Note that the patch >> requires Kaigai-san's custom foriegn join patch[1] > >

Re: [HACKERS] Join push-down support for foreign tables

2014-12-15 Thread Shigeru Hanada
remote might require whole planner to run for the join subtree. Regards, -- Shigeru HANADA join_pushdown.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Typo in bgworker.sgml

2014-10-13 Thread Shigeru Hanada
I found a minor typo in bgworker.sgml. Patch attached. It should be applied to 9.4 too. -- Shigeru HANADA fix_typo_in_bgworker.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] Join push-down support for foreign tables

2014-10-03 Thread Shigeru Hanada
2014-09-08 8:07 GMT+09:00 Shigeru HANADA : > (2014/09/04 21:37), Robert Haas wrote:> On Wed, Sep 3, 2014 at 5:16 AM, >> Probably both the initial cost and final cost calculations should be >> delegated to the FDW, but maybe within postgres_fdw, the initial cost >> should d

Re: [HACKERS] Join push-down support for foreign tables

2014-09-07 Thread Shigeru HANADA
che some statistics on the local side. But such statistics would have FDW-specific shape so it would be hard to have API to manage. FDW can have their own functions and tables to manage their own statistics, and it can have even background-worker for messaging. But it would be another story. Regards, -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Join push-down support for foreign tables

2014-09-07 Thread Shigeru HANADA
(2014/09/04 21:37), Robert Haas wrote:> On Wed, Sep 3, 2014 at 5:16 AM, Shigeru Hanada wrote: >> (1) Separate cost estimation phases? >> For existing join paths, planner estimates their costs in two phaeses. >> In the first phase initial_cost_foo(), here foo is one of

[HACKERS] Join push-down support for foreign tables

2014-09-03 Thread Shigeru Hanada
FDWs, but foreign join is not limited to such FDWs, e.g. file_fdw can have extra file which is already joined files accessed via foreign tables. -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2014-09-01 Thread Shigeru Hanada
s not expected that postgresReScanForeignScan is called when the >>> UPDATE/DELETE >>> is pushed down, right? Maybe it would make sense to add an assertion >>> for that. >> >> >> IIUC, that is right. As ModifyTable doesn't support rescan current

Re: Compute attr_needed for child relations (was Re: [HACKERS] inherit support for foreign tables)

2014-08-26 Thread Shigeru Hanada
that it >> would change, but it will be better to stick to comments in RelOptInfo >> 443 AttrNumber min_attr; /* smallest attrno of rel (often >> <0) */ >> 444 AttrNumber max_attr; /* largest attrno of rel */ >> 445 Relids *attr_needed;/* array indexed [min_attr .. >> max_attr] */ > > > Good point! Attached is the revised version of the patch. > > > Thanks, > > Best regards, > Etsuro Fujita -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2014-08-12 Thread Shigeru Hanada
-down for SELECT, UPDATE and DELETE. 2014-08-04 20:30 GMT+09:00 Etsuro Fujita : > (2014/07/30 17:22), Etsuro Fujita wrote: >> >> (2014/07/29 0:58), Robert Haas wrote: >>> >>> On Fri, Jul 25, 2014 at 3:39 AM, Albe Laurenz >>> wrote: >>>> >>&

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2014-08-04 Thread Shigeru Hanada
Hi Fujita-san, Here is a new review result from Eitoku-san. 2014-07-25 16:30 GMT+09:00 Etsuro Fujita : > (2014/07/24 18:30), Shigeru Hanada wrote: > I'm not sure that I understand your question correctly, but the reason for > that is because foreign tables cannot have INSTEAD OF tr

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2014-07-24 Thread Shigeru Hanada
of fdw_private elements is not a ranged value but an enum value (I mean that fdw_private holds 3 or 7, not 4~6, so Max and Min seems inappropriate for prefix. IMO those macros should be named so that the names represent the behavior, or define a macro to determine the mode, say IS_SHIPPABLE(foo).

Re: [HACKERS] [v9.5] Custom Plan API

2014-07-15 Thread Shigeru Hanada
for committer". -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [v9.5] Custom Plan API

2014-07-15 Thread Shigeru Hanada
ed, but the patch fails sanity-check test, you need to modify expected file of the test. > Nothing to comment on the design and implementation from your > viewpoint any more? As much as I can tell, the design seems reasonable. After fix for the small issue above, I'll move the p

Re: [HACKERS] Incorrect comment in postgres_fdw.c

2014-07-14 Thread Shigeru Hanada
this function *on > success* in that function. > > (I guess the comment has been copied and pasted from that for > get_remote_estimate().) +1 for just removing the comment, because header comment clearly mentions necessity of releasing the PGresult. -- Shigeru HANADA -- Sent

Re: [HACKERS] [v9.5] Custom Plan API

2014-07-14 Thread Shigeru Hanada
d, but a few structures still needs to have > forward declarations. > * Fix typos being pointed out. Check. I found some typos and a wording "datatype" which is not used in any other place. Please refer the attached patch. -- Shigeru HANADA fix_typo_in_v3.patch Description: Bi

Re: [HACKERS] inherit support for foreign tables

2014-07-10 Thread Shigeru Hanada
is accessed via parent table, but it sounds strange to me, because one of main purposes of tableoid is determine actual source table in appended results. Am I missing the point? -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [v9.5] Custom Plan API

2014-07-03 Thread Shigeru Hanada
O!) hope documents about writing a custom plan provider. Regards, 2014-06-17 23:12 GMT+09:00 Kohei KaiGai : > Hanada-san, > > Thanks for your checks. I oversight the points when I submit the patch, sorry. > The attached one is revised one on documentation stuff and contrib/M

Re: [HACKERS] pg_stat directory and pg_stat_statements

2014-06-19 Thread Shigeru Hanada
Fujii-san, I found the right description in REL9_3_STABLE branch, thanks for pointing out the commit. Sorry for noise. 2014-06-18 12:39 GMT+09:00 Fujii Masao : > On Tue, Jun 17, 2014 at 2:11 PM, Shigeru Hanada > wrote: >> Fujii-san, >> >> I agree not to backpatch, bu

Re: [HACKERS] pg_stat directory and pg_stat_statements

2014-06-16 Thread Shigeru Hanada
were collected in past would disappear. OTOH, the server can >>> keep >>> running successfully from then and no critical data will not >>> disappear. Therefore >>> I think we can commit this patch even at beta1. Thought? >> >> For HEAD, probably y

Re: [HACKERS] [v9.5] Custom Plan API

2014-06-16 Thread Shigeru Hanada
cement of FDW is not a way to go >> 2. Custom-path that can replace built-in scan makes sense as a first step >>towards the future enhancement. Its planner integration is enough simple >>to do. >> 3. Custom-path that can replace built-in join takes investigation

Re: [HACKERS] postgres_fdw and connection management

2014-05-26 Thread Shigeru Hanada
he pool completely as a server-specific configuration. Connection management seems FDW-specific feature to me. How about to add FDW option, say pool_connection=true|false, to postgres_fdw which allows per-server configuration? -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-h

[HACKERS] fix worker_spi to run as non-dynamic background worker

2014-05-23 Thread Shigeru Hanada
dynamic background workers can request notification LOG: redirecting log output to logging collector process HINT: Future log output will appear in directory "pg_log". Attached patch fixes this issue. Please apply onto HEAD and 9.4. -- Shigeru HANADA fix_worker_spi.patch Descrip

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-07 Thread Shigeru Hanada
primarily, > however, things to do is similar. We may need to revisit the Hanada-san's > proposition in the past. I can't recall the details soon but the reason I gave up was about introducing ForiegnJoinPath node, IIRC. I'll revisit the discussion and my proposal. -- Shigeru

Re: Custom Scan APIs (Re: [HACKERS] Custom Plan node)

2014-02-26 Thread Shigeru Hanada
like, but at least you need to be > able to return multiple paths for a single joinrel, and should look at > fixing the PlanForeignScan API to allow that too. And I've gave up for lack of time, IOW to finish more fundamental portion of FDW API. http://www.postgresql.org/message-id/4f

Re: Custom Scan APIs (Re: [HACKERS] Custom Plan node)

2014-02-26 Thread Shigeru Hanada
es that, some typo fixing are mixed in part-2 patch. They should go into the part-1 patch where the typo introduced. -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: Custom Scan APIs (Re: [HACKERS] Custom Plan node)

2014-02-25 Thread Shigeru Hanada
y avoided calling create_customscan_path. -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: Custom Scan APIs (Re: [HACKERS] Custom Plan node)

2014-02-25 Thread Shigeru Hanada
hose internal functions. If (at the moment) those information is enough for overriding merge join for CSP, let's provide as parameters. -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: Custom Scan APIs (Re: [HACKERS] Custom Plan node)

2014-02-24 Thread Shigeru Hanada
y, small amount of discussion we could have in this commit > fest, even though Hanada-san volunteered to move the patches into > "ready for committer" state at the CF-Nov. I found some cosmetic flaw and .gitignore leak in the patches. Please see attached a patc

Re: Custom Scan APIs (Re: [HACKERS] Custom Plan node)

2014-02-24 Thread Shigeru Hanada
it should be documented how to use those parameters. IMHO, minimal interface seems better than fully-fledged but half-baked one, especially in the first-cut. -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] inherit support for foreign tables

2014-02-18 Thread Shigeru Hanada
y. Changing data type from compatible one (i.e., int to numeric, varchar(n) to text), adding CHECK/NOT NULL constraint would be also possible. -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.o

Re: [HACKERS] inherit support for foreign tables

2014-02-17 Thread Shigeru Hanada
we can't do same thing as 2), because it is not a child anymore. So IMO we should determine which ALTER TABLE features are allowed to foreign tables, and allow them regardless of the recursivity. Comments? -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hacker

Re: [HACKERS] inherit support for foreign tables

2014-01-29 Thread Shigeru Hanada
(2014/01/27 21:52), Shigeru Hanada wrote: > 2014-01-27 Etsuro Fujita : >> While still reviwing this patch, I feel this patch has given enough >> consideration to interactions with other commands, but found the following >> incorrect? behabior: >> >> postgres=#

Re: [HACKERS] inherit support for foreign tables

2014-01-27 Thread Shigeru Hanada
duct1" is not a table or materialized view > > ISTN the ALTER TABLE simple recursion mechanism (ie ATSimpleRecursion()) > should be modified for the ALTER COLUMN SET STORAGE case. > > I just wanted to quickly tell you this for you to take time to consider. Thanks for the review.

Re: [HACKERS] [Review] inherit support for foreign tables

2014-01-27 Thread Shigeru Hanada
2014-01-27 Etsuro Fujita : > (2014/01/25 11:27), Shigeru Hanada wrote: > Yeah, the consistency is essential for its ease of use. But I'm not sure > that inherited stats ignoring foreign tables is actually useful for query > optimization. What I think about the consistency

Re: [HACKERS] [Review] inherit support for foreign tables

2014-01-24 Thread Shigeru Hanada
Hi Fujita-san, Thanks for the review. 2014/1/23 Etsuro Fujita : > Shigeru Hanada wrote: >> Though this would be debatable, in current implementation, constraints >> defined on a foreign table (now only NOT NULL and CHECK are supported) >> are not enforced during INSERT or UP

Re: [HACKERS] inherit support for foreign tables

2014-01-20 Thread Shigeru Hanada
1) ASSERTIVE, … CONSTRAINT chk_foo_name_upper CHECK (upper(name) = name) ASSERTIVE ) SERVER server; BTW, I noticed that this is like push-down-able expressions in JOIN/WHERE. We need to check a CHECK constraint defined on a foreign tables contains only expressions which have s

Re: [HACKERS] inherit support for foreign tables

2014-01-14 Thread Shigeru Hanada
Hi all, 2014/1/14 Shigeru Hanada : > I'd like to revisit this feature. Attached patch allows a foreign table to be a child of a table. It also allows foreign tables to have CHECK constraints. These changes provide us a chance to propagate query load to multiple servers via constraint e

Re: [HACKERS] inherit support for foreign tables

2014-01-14 Thread Shigeru Hanada
CK constraints, currently NOT NULL constraints are virtually ASSERTIVE (not enforcing). Should it also be noted explicitly? Thoughts? -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: Custom Scan APIs (Re: [HACKERS] Custom Plan node)

2013-12-16 Thread Shigeru Hanada
KaiGai-san, 2013/12/16 KaiGai Kohei : > (2013/12/16 14:15), Shigeru Hanada wrote: >> (1) ctidscan >> Is session_preload_libraries available to enable the feature, like >> shared_*** and local_***? According to my trial it works fine like >> two similar GUCs. >>

Re: Custom Scan APIs (Re: [HACKERS] Custom Plan node)

2013-12-15 Thread Shigeru Hanada
Hi Kaigai-san, 2013/12/11 Kohei KaiGai : > 2013/12/10 Shigeru Hanada : >> The patches could be applied cleanly, but I saw a compiler warning >> about get_rel_relkind() in foreign.c, but it's minor issue. Please >> just add #include of utils/lsyscache.h there. >>

Re: Custom Scan APIs (Re: [HACKERS] Custom Plan node)

2013-12-10 Thread Shigeru Hanada
ames used in the join or the scan would help users to understand what is going on in Custom Scan. Regards, -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: Custom Scan APIs (Re: [HACKERS] Custom Plan node)

2013-12-03 Thread Shigeru Hanada
2013/11/29 Kohei KaiGai : > I merged all the propositions from Jim. Thanks, it made the documentation > quality better. Also, I fixed up cosmetic stuff around whitespace <-> tab. I found some typos in documents and comments. Please see attached patch for detail. -- Sh

Re: Custom Scan APIs (Re: [HACKERS] Custom Plan node)

2013-12-03 Thread Shigeru Hanada
lation and has_wholerow_reference seem to be useful for other FDWs. Is it able to move them into core? (4) postgres_fdw estimates costs of join by calculating local numbers. How about to support remote estimation by throwing EXPLALAIN query when use_remote_estimates = true. -- Shigeru HANA

Re: [HACKERS] Status of FDW pushdowns

2013-11-27 Thread Shigeru Hanada
low new syntax like WITH alias AS FOREIGN VIEW (remote query)? I think it's nice to support executing ad-hoc remote query written in the syntax which is valid only on remote data source through FDW, and at the moment dblink interface seems feasible for that purpose. -- Shigeru HANADA --

Re: [HACKERS] Status of FDW pushdowns

2013-11-27 Thread Shigeru Hanada
Hi Merlin, 2013/11/22 Merlin Moncure : > On Thu, Nov 21, 2013 at 6:43 PM, Shigeru Hanada > wrote: >> 2013/11/22 Tom Lane : >>> Merlin Moncure writes: >>>> On Thu, Nov 21, 2013 at 9:05 AM, Bruce Momjian wrote: >>>>> I know join pushdowns s

Re: Custom Scan APIs (Re: [HACKERS] Custom Plan node)

2013-11-26 Thread Shigeru Hanada
commit 784e762e886e6f72f548da86a27cd2ead87dbd1c (committed on 2013-11-21) allows FunctionScan node to have multiple function expression, so Node * funcexpr in CustomScan should be List *funcitons now. I'll continue to review by applying patches onto 2013-11-19 HEAD. -- Shigeru HANADA -- Sen

Re: [HACKERS] Status of FDW pushdowns

2013-11-21 Thread Shigeru Hanada
use dblink. For better or worse, the FDW stuff is > following the SQL standard's SQL/MED design, which does not do it > like that. Pass-through mode mentioned in SQL/MED standard might be what he wants. -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Status of FDW pushdowns

2013-11-21 Thread Shigeru Hanada
define "foreign index" on foreign tables to indicate which column combination is reasonably sortable. -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] inherit support for foreign tables

2013-11-13 Thread Shigeru Hanada
ign table as a child node of Append Currently prepunion.c assumes that children of Append have RELKIND_RELATION as relkind always, so we need to set relkind of child RTE explicitly. Please see attached PoC patch. I'll enhance implementation, tests and document and submit the patch for the next

Re: [HACKERS] Writable foreign tables: how to identify rows

2013-03-06 Thread Shigeru Hanada
I'm not sure whether postgres_fdw should support, but updatable views have no system column including ctid. So, we need magic identifier, perhaps it would be set of primary key value(s), to support updating remote updatable views via foreign tables. Just a heads up. -- Shigeru HANADA -- Sen

Re: [HACKERS] Writable foreign tables: how to identify rows

2013-03-06 Thread Shigeru Hanada
mine source server. -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] FDW for PostgreSQL

2013-02-20 Thread Shigeru Hanada
, but replace one if FDW option *_name was specified for an object. With this context, existing ruleutils would generate expression-strings with proper names, without any change. Is this idea acceptable? -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)

Re: [HACKERS] FDW for PostgreSQL

2013-02-16 Thread Shigeru Hanada
'bid') | plain| | bbalance | integer | | | plain| | filler | character(88) | | | extended | | Server: pgbench1 FDW Options: (schema 'public', "table" 'foo&

Re: [HACKERS] FDW for PostgreSQL

2013-02-14 Thread Shigeru Hanada
On Thu, Feb 14, 2013 at 6:45 PM, Albe Laurenz wrote: > Shigeru Hanada wrote: >> Tom Lane wrote: >>> It ought to be pulling the rows back a few at a time, and >>> that's not going to work well if multiple scans are sharing the same >>> connec

Re: [HACKERS] FDW for PostgreSQL

2013-02-14 Thread Shigeru Hanada
on ruleutils.c > for reverse-listing services, and it would be best to bite that bullet > now and rewrite this code from scratch. I thought that writing ruleutils-free SQL constructor in postgres_fdw is better approach because ruleutils might be changed from its own purpose in future. Besides that, as Kaigai-san mentioned in upthread, ruleutils seems to have insufficient capability for building remote PostgreSQL query. -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] proposal - assign result of query to psql variable

2013-02-02 Thread Shigeru Hanada
ullPrint > d) use empty String +1 for a). If users want to determine whether the result was NULL, or want to use substitute string for NULL result, they can use coalesce in SELECT clause. Anyway the feature should be documented clearly. -- Shigeru HANADA -- Sent via pgsql-hackers m

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2013-01-08 Thread Shigeru Hanada
ixed palloc call. OK, both threshold and initial palloc were fixed correctly. I'll mark this patch as "Ready for committer". Good work! :-) Regards, -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2013-01-07 Thread Shigeru Hanada
cordance with the > previous discussion. New name sounds good to me, but the #define says that the value is 15. Should it be fixed to 20? >> * +1 for Alvaro's suggestion about avoiding palloc traffic by starting >> with 8 elements or so. >> > > Done. Not yet. Init

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2013-01-07 Thread Shigeru Hanada
_SEARCH_LIMIT? # using RELS instead of RELATIONS would be better to shorten the name * +1 for Alvaro's suggestion about avoiding palloc traffic by starting with 8 elements or so. Regards, -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To ma

Re: [HACKERS] proposal - assign result of query to psql variable

2012-12-18 Thread Shigeru Hanada
On Tue, Dec 18, 2012 at 2:52 AM, Pavel Stehule wrote: > Attached updated patch Seems fine. I'll mark this as "ready for committer". Nice work! -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] proposal - assign result of query to psql variable

2012-12-17 Thread Shigeru Hanada
ee attached patch for details. * How about pulling up codes for PGRES_TUPLES_OK case in StoreQueryResult to new static function, say StoreQueryTuple? It would make StoreQueryResult more similar to PrintQueryResult's style, and IMO it makes the code more readable. Regards, -- Shigeru HANADA g

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2012-12-09 Thread Shigeru Hanada
ndent variable name in declarations. For these two issues, please see the page below: http://www.postgresql.org/docs/9.2/static/source-format.html * PostgreSQL allows C89 features, so we can't use variable length array. * Contains unintentional blank lines? Please see attached patch for details o

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2012-12-05 Thread Shigeru Hanada
t; get larger differences, as we use larger shared buffers and the memory > is significantly slower there. Do you have performance numbers of same test on not-patched PG? This patch aims to improve performance of bulk DROP, so 4th number in the result above should be compared to 4th number

Re: [HACKERS] FDW for PostgreSQL

2012-11-22 Thread Shigeru Hanada
when VERBOSE was specified. This would make EXPLAIN output easy to read, even if many foreign tables are used in a query. Thoughts? -- Shigeru HANADA

Re: [HACKERS] FDW for PostgreSQL

2012-11-21 Thread Shigeru Hanada
As you say, postgres_fdw had used cursor to avoid possible memory exhaust on large result set. I switched to single-row processing mode (it could be said "protocol-level cursor"), which was added in 9.2, because it accomplish same task with less SQL calls than cursor. Regards, -- Shigeru HANADA

Re: [HACKERS] Move postgresql_fdw_validator into dblink

2012-11-14 Thread Shigeru Hanada
fdw, and I chose the one more familiar to me at that time. I think that only few people feel that "postgres" is shortened name of postgresql. How about using postgres_fdw for PG-FDW? Regards, -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] FDW for PostgreSQL

2012-11-06 Thread Shigeru HANADA
mentioning it clearly might be better. Regards, -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Move postgresql_fdw_validator into dblink

2012-10-30 Thread Shigeru Hanada
fusing. In addition, it would be worth mentioning that it's not recommended to use postgresql_fdw_validator as validator of a third-party's FDW to avoid dependency. Regards, -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

  1   2   3   >