Re: [HACKERS] [v9.5] Custom Plan API

2014-06-16 Thread Shigeru Hanada
cement of FDW is not a way to go >> 2. Custom-path that can replace built-in scan makes sense as a first step >>towards the future enhancement. Its planner integration is enough simple >>to do. >> 3. Custom-path that can replace built-in join takes investigation

Re: [HACKERS] pg_stat directory and pg_stat_statements

2014-06-16 Thread Shigeru Hanada
were collected in past would disappear. OTOH, the server can >>> keep >>> running successfully from then and no critical data will not >>> disappear. Therefore >>> I think we can commit this patch even at beta1. Thought? >> >> For HEAD, probably y

Re: [HACKERS] pg_stat directory and pg_stat_statements

2014-06-19 Thread Shigeru Hanada
Fujii-san, I found the right description in REL9_3_STABLE branch, thanks for pointing out the commit. Sorry for noise. 2014-06-18 12:39 GMT+09:00 Fujii Masao : > On Tue, Jun 17, 2014 at 2:11 PM, Shigeru Hanada > wrote: >> Fujii-san, >> >> I agree not to backpatch, bu

Re: [HACKERS] [v9.5] Custom Plan API

2014-07-03 Thread Shigeru Hanada
O!) hope documents about writing a custom plan provider. Regards, 2014-06-17 23:12 GMT+09:00 Kohei KaiGai : > Hanada-san, > > Thanks for your checks. I oversight the points when I submit the patch, sorry. > The attached one is revised one on documentation stuff and contrib/M

Re: [HACKERS] inherit support for foreign tables

2014-07-10 Thread Shigeru Hanada
is accessed via parent table, but it sounds strange to me, because one of main purposes of tableoid is determine actual source table in appended results. Am I missing the point? -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [v9.5] Custom Plan API

2014-07-14 Thread Shigeru Hanada
d, but a few structures still needs to have > forward declarations. > * Fix typos being pointed out. Check. I found some typos and a wording "datatype" which is not used in any other place. Please refer the attached patch. -- Shigeru HANADA fix_typo_in_v3.patch Description: Bi

Re: [HACKERS] Incorrect comment in postgres_fdw.c

2014-07-14 Thread Shigeru Hanada
this function *on > success* in that function. > > (I guess the comment has been copied and pasted from that for > get_remote_estimate().) +1 for just removing the comment, because header comment clearly mentions necessity of releasing the PGresult. -- Shigeru HANADA -- Sent

Re: [HACKERS] [v9.5] Custom Plan API

2014-07-15 Thread Shigeru Hanada
ed, but the patch fails sanity-check test, you need to modify expected file of the test. > Nothing to comment on the design and implementation from your > viewpoint any more? As much as I can tell, the design seems reasonable. After fix for the small issue above, I'll move the p

Re: [HACKERS] [v9.5] Custom Plan API

2014-07-15 Thread Shigeru Hanada
for committer". -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2014-07-24 Thread Shigeru Hanada
of fdw_private elements is not a ranged value but an enum value (I mean that fdw_private holds 3 or 7, not 4~6, so Max and Min seems inappropriate for prefix. IMO those macros should be named so that the names represent the behavior, or define a macro to determine the mode, say IS_SHIPPABLE(foo).

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2014-08-04 Thread Shigeru Hanada
Hi Fujita-san, Here is a new review result from Eitoku-san. 2014-07-25 16:30 GMT+09:00 Etsuro Fujita : > (2014/07/24 18:30), Shigeru Hanada wrote: > I'm not sure that I understand your question correctly, but the reason for > that is because foreign tables cannot have INSTEAD OF tr

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2014-08-12 Thread Shigeru Hanada
-down for SELECT, UPDATE and DELETE. 2014-08-04 20:30 GMT+09:00 Etsuro Fujita : > (2014/07/30 17:22), Etsuro Fujita wrote: >> >> (2014/07/29 0:58), Robert Haas wrote: >>> >>> On Fri, Jul 25, 2014 at 3:39 AM, Albe Laurenz >>> wrote: >>>> >>&

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2015-08-04 Thread Shigeru Hanada
gt; Rest of the patch looks fine. Thanks -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] inherit support for foreign tables

2014-01-14 Thread Shigeru Hanada
CK constraints, currently NOT NULL constraints are virtually ASSERTIVE (not enforcing). Should it also be noted explicitly? Thoughts? -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] inherit support for foreign tables

2014-01-14 Thread Shigeru Hanada
Hi all, 2014/1/14 Shigeru Hanada : > I'd like to revisit this feature. Attached patch allows a foreign table to be a child of a table. It also allows foreign tables to have CHECK constraints. These changes provide us a chance to propagate query load to multiple servers via constraint e

Re: [HACKERS] inherit support for foreign tables

2014-01-20 Thread Shigeru Hanada
1) ASSERTIVE, … CONSTRAINT chk_foo_name_upper CHECK (upper(name) = name) ASSERTIVE ) SERVER server; BTW, I noticed that this is like push-down-able expressions in JOIN/WHERE. We need to check a CHECK constraint defined on a foreign tables contains only expressions which have s

Re: [HACKERS] [Review] inherit support for foreign tables

2014-01-24 Thread Shigeru Hanada
Hi Fujita-san, Thanks for the review. 2014/1/23 Etsuro Fujita : > Shigeru Hanada wrote: >> Though this would be debatable, in current implementation, constraints >> defined on a foreign table (now only NOT NULL and CHECK are supported) >> are not enforced during INSERT or UP

Re: [HACKERS] [Review] inherit support for foreign tables

2014-01-27 Thread Shigeru Hanada
2014-01-27 Etsuro Fujita : > (2014/01/25 11:27), Shigeru Hanada wrote: > Yeah, the consistency is essential for its ease of use. But I'm not sure > that inherited stats ignoring foreign tables is actually useful for query > optimization. What I think about the consistency

Re: [HACKERS] inherit support for foreign tables

2014-01-27 Thread Shigeru Hanada
duct1" is not a table or materialized view > > ISTN the ALTER TABLE simple recursion mechanism (ie ATSimpleRecursion()) > should be modified for the ALTER COLUMN SET STORAGE case. > > I just wanted to quickly tell you this for you to take time to consider. Thanks for the review.

Re: [HACKERS] inherit support for foreign tables

2014-01-29 Thread Shigeru Hanada
(2014/01/27 21:52), Shigeru Hanada wrote: > 2014-01-27 Etsuro Fujita : >> While still reviwing this patch, I feel this patch has given enough >> consideration to interactions with other commands, but found the following >> incorrect? behabior: >> >> postgres=#

Re: [HACKERS] inherit support for foreign tables

2014-02-17 Thread Shigeru Hanada
we can't do same thing as 2), because it is not a child anymore. So IMO we should determine which ALTER TABLE features are allowed to foreign tables, and allow them regardless of the recursivity. Comments? -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hacker

Re: [HACKERS] inherit support for foreign tables

2014-02-18 Thread Shigeru Hanada
y. Changing data type from compatible one (i.e., int to numeric, varchar(n) to text), adding CHECK/NOT NULL constraint would be also possible. -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.o

Re: Custom Scan APIs (Re: [HACKERS] Custom Plan node)

2014-02-24 Thread Shigeru Hanada
it should be documented how to use those parameters. IMHO, minimal interface seems better than fully-fledged but half-baked one, especially in the first-cut. -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: Custom Scan APIs (Re: [HACKERS] Custom Plan node)

2014-02-24 Thread Shigeru Hanada
y, small amount of discussion we could have in this commit > fest, even though Hanada-san volunteered to move the patches into > "ready for committer" state at the CF-Nov. I found some cosmetic flaw and .gitignore leak in the patches. Please see attached a patc

Re: Custom Scan APIs (Re: [HACKERS] Custom Plan node)

2014-02-25 Thread Shigeru Hanada
hose internal functions. If (at the moment) those information is enough for overriding merge join for CSP, let's provide as parameters. -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: Custom Scan APIs (Re: [HACKERS] Custom Plan node)

2014-02-25 Thread Shigeru Hanada
y avoided calling create_customscan_path. -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: Custom Scan APIs (Re: [HACKERS] Custom Plan node)

2014-02-26 Thread Shigeru Hanada
es that, some typo fixing are mixed in part-2 patch. They should go into the part-1 patch where the typo introduced. -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: Custom Scan APIs (Re: [HACKERS] Custom Plan node)

2014-02-26 Thread Shigeru Hanada
like, but at least you need to be > able to return multiple paths for a single joinrel, and should look at > fixing the PlanForeignScan API to allow that too. And I've gave up for lack of time, IOW to finish more fundamental portion of FDW API. http://www.postgresql.org/message-id/4f

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-02-01 Thread Shigeru Hanada
ction at the end of each local query. I chose this timing because local query might include multiple scans on same foreign server. I think this would be "ASAP" timing in your comment. It would be useful to make length of remote transaction same as local's, I'll try Register

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-02-05 Thread Shigeru Hanada
2012/2/2 Marko Kreen : I think you want this instead: Â https://commitfest.postgresql.org/action/patch_view?id=769 Somehow I've missed this cool feature. Thanks for the suggestion! -- Shigeru Hanada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make chang

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-02-06 Thread Shigeru Hanada
t logic to determine whether qualifier > is available to push down, or not. > > For example, it checks oid of operator and oid of function being > invoked on this operator at OpExpr. > However, the purpose of this check is to ensure same result on > execution of qualifier, thus it r

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-02-07 Thread Shigeru Hanada
e better to describe how to determine whether a column result is NULL should be described clearly. Perhaps result value is NULL when PGrowValue.len is less than 0, right? Regards, -- Shigeru Hanada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-02-07 Thread Shigeru Hanada
(2012/02/07 23:44), Marko Kreen wrote: > On Tue, Feb 07, 2012 at 07:25:14PM +0900, Shigeru Hanada wrote: >> - What is the right (or recommended) way to prevent from throwing >> exceptoin in row-processor callback function? When author should use >> PG_TRY block to catch exce

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-02-08 Thread Shigeru Hanada
l_fdw(direct access) : 7719 - 6.3x faster than Row Processor Row processor was almost 2x faster than SQL-cursor! I'm looking forward to this feature. In addition to performance gain, of course memory usage was kept at very low level. :) Regards, -- Shigeru Hanada -- Sent via pgsql-hac

Re: [HACKERS] WIP: Collecting statistics on CSV file data

2012-02-10 Thread Shigeru Hanada
(2011/12/15 11:30), Etsuro Fujita wrote: > (2011/12/14 15:34), Shigeru Hanada wrote: >> I think this patch could be marked as "Ready for committer" with some >> minor fixes. Please find attached a revised patch (v6.1). I've tried to make pgsql_fdw work with th

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-02-13 Thread Shigeru Hanada
after 9.2 development enters beta or RC, hopefully :) -- Shigeru Hanada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-02-14 Thread Shigeru Hanada
o achieve them for 9.2. Regards, -- Shigeru Hanada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-02-16 Thread Shigeru Hanada
(2012/02/17 0:15), Albe Laurenz wrote: > Shigeru Hanada wrote: >> Thanks for the review. Attached patches are revised version, though >> only fdw_helper_v5.patch is unchanged. > > Two questions: > - Is it on purpose that you can specify all SSL client options >e

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-02-20 Thread Shigeru Hanada
2012/02/21 0:58 "Kevin Grittner" : > > "Albe Laurenz" wrote: > > > I read the example carefully, and it seems to me that it is > > necessary for the read-only transaction (T3)v to be SERIALIZABLE so > > that T1 is aborted and the state that T3 saw remains valid. > > Correct. Hm, agreed that isolat

Re: [HACKERS] How to know a table has been modified?

2012-02-27 Thread Shigeru Hanada
reading archived WAL files? They would contain every delete/update/drop including cascading ones, though it might be too late because WAL files are not available until archived. xlogdump would help reading WAL files. https://github.com/snaga/xlogdump -- Shigeru Hanada -- Sent via pgsql-ha

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-02-27 Thread Shigeru Hanada
_fdw_validator, which is implemented in backend binary and used by contrib/dblink. I thought that we should use another name for PostgreSQL FDW unless we can change specification of dblink connection string. -- Shigeru Hanada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)

Re: [HACKERS] FDW system columns

2012-02-28 Thread Shigeru Hanada
rit other tables. With such feature, tableoid of foreign table is necessary to determine actual source table. Once we want to support that feature, IMO we should revive tableoid system column for foreign tables. -- Shigeru Hanada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] FDW system columns

2012-02-28 Thread Shigeru Hanada
all system columns as is (current 9.2dev) Incidentally, views, which is very similar object type to foreign tables, have no system columns. Thoughts? -- Shigeru Hanada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] FDW system columns

2012-02-28 Thread Shigeru Hanada
(2012/02/28 23:37), Kohei KaiGai wrote: > 2012年2月28日12:00 Shigeru Hanada: >> We have three options: >> >> a) remove all system columns (posted patch) >> b) remove system columns other than tableoid >> c) leave all system columns as is (current 9.2dev) >>

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-02-29 Thread Shigeru Hanada
new libpq function, like below, which returns 1 when given name is a valid libpq option would help. int PQisValidOption(const char *keyword); -- Shigeru Hanada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-03-01 Thread Shigeru Hanada
l_fdw to postgresql_fdw which contains product name fully spelled out. -- Shigeru Hanada diff --git a/contrib/dblink/Makefile b/contrib/dblink/Makefile index ac63748..a27db88 100644 *** a/contrib/dblink/Makefile --- b/contrib/dblink/Makefile *** SHLIB_LINK = $(libpq) *** 7,13 SHL

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-03-01 Thread Shigeru Hanada
uot; in the name of validator which doesn't have actual FDW might confuse users. Rather dblink_validator or libpq_option_validator is better? One possible another idea is creating dblink_fdw which uses the validator during "CREATE EXTENSION dblink" for users who store connecti

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-03-05 Thread Shigeru Hanada
s restriction wouldn't be a serious problem. Please find attached a patch implementing the design above. -- Shigeru Hanada diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c index 46394a8..bb541e3 100644 *** a/contrib/file_fdw/file_fdw.c --- b/contrib/f

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-03-05 Thread Shigeru Hanada
m sorry about that. Perhaps you are confusing create_index_path with create_index_paths. Former creates a IndexScan path node (so it's similar to create_foreignscan_path), and latter builds multiple IndexScan paths for a plain relation. So, just renaming create_foreignscan_path to plural f

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-03-06 Thread Shigeru Hanada
e FDW authors check. Regards, -- Shigeru Hanada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] missing description initdb manual

2012-03-09 Thread Shigeru Hanada
in document. -- Shigeru Hanada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-03-09 Thread Shigeru Hanada
scan_path for the query. GetForeignPlan 1) Create fdw_exprs from baserestrictinfo, with removing clauses which are pushed down by selected path. Regards, -- Shigeru Hanada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-03-09 Thread Shigeru Hanada
SELECT and FROM, and other paths can be built by copying necessary clauses into individual buffers. Comments to the rest part are in my another reply to your recent post. Regards, -- Shigeru Hanada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-03-09 Thread Shigeru Hanada
(2012/03/06 23:47), Albe Laurenz wrote: > Shigeru Hanada wrote: >> Connection should be closed only when the trigger is a >> top level transaction and it's aborting, but isTopLevel flag was not >> checked. I fixed the bug and added regression tests for

Re: [HACKERS] NOT NULL violation error handling in file_fdw

2012-03-12 Thread Shigeru HANADA
ecking codes from ExecConstraints, and call it from fileIterateForeignScan and ExecConstraints. Anyway, please add this patch to Commit Fest App for tracking. https://commitfest.postgresql.org/action/commitfest_view?id=14 -- Shigeru Hanada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgre

[HACKERS] Why does exprCollation reject List node?

2012-03-16 Thread Shigeru HANADA
from Expr, and it rejects other objects with elog(ERROR). Anyone know the reason? Regards, -- Shigeru Hanada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Proposal: Create index on foreign table

2012-03-16 Thread Shigeru Hanada
DWs. The design you shown here seems indexed-file_fdw to me... Regards, -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Why does exprCollation reject List node?

2012-03-18 Thread Shigeru Hanada
On Fri, Mar 16, 2012 at 10:17 PM, Tom Lane wrote: > Shigeru HANADA writes: >> During writing pgsql_fdw codes, I noticed that exprCollation rejects >> non-Expr nodes with error "unrecognized node type: %d".  Is this >> intentional behavior, or can it return In

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-03-27 Thread Shigeru HANADA
wing them would cover significant cases of string comparison, but I'm not sure how to determine whether an operator is = or != in generic way. We might have to hold list of oid for collation-safe operator/functions until we support ROUTINE MAPPING or something like that... Anyway, I&#x

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-03-27 Thread Shigeru HANADA
(2012/03/27 20:32), Thom Brown wrote: > 2012/3/26 Shigeru HANADA: >> * pgsql_fdw_v17.patch >> - Adds pgsql_fdw as contrib module >> * pgsql_fdw_pushdown_v10.patch >> - Adds WHERE push down capability to pgsql_fdw >> * pgsql_fdw_analyze_v1.patch >>

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-03-28 Thread Shigeru HANADA
ly depend on statistics of foreign data, but IMO having statistics of foreign data which were calculated in the way same as local data seems still useful for estimation about local filtering. Even if we have no statistics of foreign data on local side, still we would be able to create plans on th

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-03-28 Thread Shigeru HANADA
reign tables is proposed by Fujita-san in current CF, so I'd like to push it. Regards, -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-04-06 Thread Shigeru HANADA
sampling (acquire_sample_rows) function seems handy place to hook. Regards, -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-04-06 Thread Shigeru Hanada
nk that assumption has a life expectancy measured in > months at most, and I don't want to have to try to fix every FDW when > it changes.  But I think we can easily revise the hook details to fix > that, and I'm hoping to get that done today. I'll try implementing the desig

Re: [HACKERS] WIP: Collecting statistics on CSV file data

2012-04-06 Thread Shigeru Hanada
On Sat, Apr 7, 2012 at 4:19 AM, Tom Lane wrote: > Shigeru HANADA writes: >> Just after my post, Fujita-san posted another v7 patch[1], so I merged >> v7 patches into v8 patch. > > I've committed a modified version of this, but right after pushing it > I had

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-04-08 Thread Shigeru HANADA
(2012/04/08 5:19), Thom Brown wrote: > 2012/4/7 Shigeru HANADA: >> I've updated pgsql_fdw so that it can collect statistics from foreign >> data with new FDW API. > > I notice that if you restart the remote server, the connection is > broken, but the client doesn&#

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-04-11 Thread Shigeru HANADA
others in only SERVER options (and reject all debug options). IMO this is not unreasonable assumption. Is this proposal reasonable? Any comments and questions are welcome. Regards, -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-03-16 Thread Shigeru Hanada
level) is an idea, but it would make planning time longer (and use more cycle and memory). Tom's idea sounds good for saving the path b), but I worry that whether FDW can get enough information at that timing, just before set_cheapest. It would not be good I/F if each FDW n

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-03-24 Thread Shigeru HANADA
reak > down join relations. With applying your patch, regression tests of “updatable view” failed. regression.diff contains some errors like this: ! ERROR: could not find RelOptInfo for given relids Could you check that? — Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-03-25 Thread Shigeru HANADA
r particular relid combination, and there SpecialJoinInfo and restrictlist (conditions specified in JOIN-ON and WHERE), so it seems promising for FDW cases. Though I’m not sure that it also fits custom join provider’s requirements. — Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-03-25 Thread Shigeru HANADA
2015/03/25 19:09、Kouhei Kaigai のメール: >> On Wed, Mar 25, 2015 at 3:14 PM, Shigeru HANADA >> wrote: >> Or bottom of make_join_rel(). IMO build_join_rel() is responsible for >> just building (or searching from a list) a RelOptInfo for given relids. >> After

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-03-25 Thread Shigeru HANADA
2015/03/25 18:53、Ashutosh Bapat のメール: > > > On Wed, Mar 25, 2015 at 3:14 PM, Shigeru HANADA > wrote: > > Or bottom of make_join_rel(). IMO build_join_rel() is responsible for just > building (or searching from a list) a RelOptInfo for given relids. After > th

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-03-25 Thread Shigeru HANADA
ld remote- > join query. Another idea is to pass “found” as parameter to FDW handler, and let FDW to decide to skip or not. Some of FDWs (and some of CSP?) might want to be conscious of join combination. — Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-03-25 Thread Shigeru HANADA
ze a query which accesses multiple parent tables, each is inherited by foreign tables and partitioned with identical join key, by building a path tree which joins sharded tables first, and then union those results. -- Shigeru HANADA shigeru.han...@gmail.com -- Sent via pgsql-hackers ma

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-04-14 Thread Shigeru HANADA
elation. Oops, that’s right. Attached is the revised version. I chose fully qualified name, schema.relname [alias] for the output. It would waste some cycles during planning if that is not for EXPLAIN, but it seems difficult to get a list of name of relations in ExplainForeignScan() phase, be

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-04-20 Thread Shigeru HANADA
changed to fix Vars in ps_tlist of CustomScan and ForeignScan. For this change, new BitmapSet function bms_shift_members is added. set_deparse_planstate is also changed to pass ps_tlist as namespace for deparsing. These chanes seems reasonable, so I mark this patch as “ready for committers” to hear committers' thoughts. Regards, -- Shigeru HANADA shigeru.han...@gmail.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-04-24 Thread Shigeru HANADA
-- Foreign Scan (cost=100.00..101.00 rows=50 width=716) Output: b.bid, b.bbalance, b.filler, t.tid, t.bid, t.tbalance, t.fille

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-04-27 Thread Shigeru HANADA
Foreign Scan (cost=100.00..101.00 rows=50 width=716) Output: b.bid, b.bbalance, b.filler, t.tid, t.bid, t.tbalance, t.filler Relations: (public.pgbench_branches b) CROSS JOIN (public.pgbench_tellers t) Remote SQL: SELECT l.a1, l.a2, l.a3, r.a1, r.a2, r.a3, r.a4 FROM (SELECT l.a9, l.a10, l.a11 FROM (SELECT bid a9, bbalance a10, filler a11 FROM public.pgbench_branches) l) l (a1, a2, a3) CROSS JOIN (SELECT r.a9, r.a10, r.a11, r.a12 FROM (SELECT tid a9, bid a10, tbalance a11, filler a12 FROM public.pgbench_tellers) r) r (a1, a2, a3, a4) WHERE ((l.a1 = r.a2)) (4 rows) Thoughts? Regards, -- Shigeru HANADA shigeru.han...@gmail.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2015-05-02 Thread Shigeru HANADA
Thanks for the comments. 2015/05/01 22:35、Robert Haas のメール: > On Mon, Apr 27, 2015 at 5:07 AM, Shigeru Hanada > wrote: >> 2015-04-27 11:00 GMT+09:00 Kouhei Kaigai : >>> Hanada-san, could you adjust your postgres_fdw patch according to >>> the above new (pre

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-05-12 Thread Shigeru Hanada
of a join tree. Some FDW might hope to use sub-plan generation, but we don't have any concrete use case as of now. -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-05-15 Thread Shigeru Hanada
(or at least via CSP I/F). -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: Compute attr_needed for child relations (was Re: [HACKERS] inherit support for foreign tables)

2014-08-26 Thread Shigeru Hanada
that it >> would change, but it will be better to stick to comments in RelOptInfo >> 443 AttrNumber min_attr; /* smallest attrno of rel (often >> <0) */ >> 444 AttrNumber max_attr; /* largest attrno of rel */ >> 445 Relids *attr_needed;/* array indexed [min_attr .. >> max_attr] */ > > > Good point! Attached is the revised version of the patch. > > > Thanks, > > Best regards, > Etsuro Fujita -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2014-09-01 Thread Shigeru Hanada
s not expected that postgresReScanForeignScan is called when the >>> UPDATE/DELETE >>> is pushed down, right? Maybe it would make sense to add an assertion >>> for that. >> >> >> IIUC, that is right. As ModifyTable doesn't support rescan current

[HACKERS] Join push-down support for foreign tables

2014-09-03 Thread Shigeru Hanada
FDWs, but foreign join is not limited to such FDWs, e.g. file_fdw can have extra file which is already joined files accessed via foreign tables. -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Join push-down support for foreign tables

2014-09-07 Thread Shigeru HANADA
(2014/09/04 21:37), Robert Haas wrote:> On Wed, Sep 3, 2014 at 5:16 AM, Shigeru Hanada wrote: >> (1) Separate cost estimation phases? >> For existing join paths, planner estimates their costs in two phaeses. >> In the first phase initial_cost_foo(), here foo is one of

Re: [HACKERS] Join push-down support for foreign tables

2014-09-07 Thread Shigeru HANADA
che some statistics on the local side. But such statistics would have FDW-specific shape so it would be hard to have API to manage. FDW can have their own functions and tables to manage their own statistics, and it can have even background-worker for messaging. But it would be another story. Regards, -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Join push-down support for foreign tables

2014-10-03 Thread Shigeru Hanada
2014-09-08 8:07 GMT+09:00 Shigeru HANADA : > (2014/09/04 21:37), Robert Haas wrote:> On Wed, Sep 3, 2014 at 5:16 AM, >> Probably both the initial cost and final cost calculations should be >> delegated to the FDW, but maybe within postgres_fdw, the initial cost >> should d

[HACKERS] Typo in bgworker.sgml

2014-10-13 Thread Shigeru Hanada
I found a minor typo in bgworker.sgml. Patch attached. It should be applied to 9.4 too. -- Shigeru HANADA fix_typo_in_bgworker.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] FDW for PostgreSQL

2012-11-21 Thread Shigeru Hanada
As you say, postgres_fdw had used cursor to avoid possible memory exhaust on large result set. I switched to single-row processing mode (it could be said "protocol-level cursor"), which was added in 9.2, because it accomplish same task with less SQL calls than cursor. Regards, -- Shigeru HANADA

Re: [HACKERS] FDW for PostgreSQL

2012-11-22 Thread Shigeru Hanada
when VERBOSE was specified. This would make EXPLAIN output easy to read, even if many foreign tables are used in a query. Thoughts? -- Shigeru HANADA

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2012-12-05 Thread Shigeru Hanada
t; get larger differences, as we use larger shared buffers and the memory > is significantly slower there. Do you have performance numbers of same test on not-patched PG? This patch aims to improve performance of bulk DROP, so 4th number in the result above should be compared to 4th number

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2012-12-09 Thread Shigeru Hanada
ndent variable name in declarations. For these two issues, please see the page below: http://www.postgresql.org/docs/9.2/static/source-format.html * PostgreSQL allows C89 features, so we can't use variable length array. * Contains unintentional blank lines? Please see attached patch for details o

Re: [HACKERS] proposal - assign result of query to psql variable

2012-12-17 Thread Shigeru Hanada
ee attached patch for details. * How about pulling up codes for PGRES_TUPLES_OK case in StoreQueryResult to new static function, say StoreQueryTuple? It would make StoreQueryResult more similar to PrintQueryResult's style, and IMO it makes the code more readable. Regards, -- Shigeru HANADA g

Re: [HACKERS] proposal - assign result of query to psql variable

2012-12-18 Thread Shigeru Hanada
On Tue, Dec 18, 2012 at 2:52 AM, Pavel Stehule wrote: > Attached updated patch Seems fine. I'll mark this as "ready for committer". Nice work! -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2013-01-07 Thread Shigeru Hanada
_SEARCH_LIMIT? # using RELS instead of RELATIONS would be better to shorten the name * +1 for Alvaro's suggestion about avoiding palloc traffic by starting with 8 elements or so. Regards, -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To ma

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2013-01-07 Thread Shigeru Hanada
cordance with the > previous discussion. New name sounds good to me, but the #define says that the value is 15. Should it be fixed to 20? >> * +1 for Alvaro's suggestion about avoiding palloc traffic by starting >> with 8 elements or so. >> > > Done. Not yet. Init

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2013-01-08 Thread Shigeru Hanada
ixed palloc call. OK, both threshold and initial palloc were fixed correctly. I'll mark this patch as "Ready for committer". Good work! :-) Regards, -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription

Re: [HACKERS] SQL/MED estimated time of arrival?

2010-11-24 Thread Shigeru HANADA
On Tue, 23 Nov 2010 10:22:02 -0800 "Joshua D. Drake" wrote: > On Tue, 2010-11-23 at 20:18 +0200, Heikki Linnakangas wrote: > > On 23.11.2010 14:22, Shigeru HANADA wrote: > > > > OID is supported to get oid from the source table (yes, it works only > > > f

[HACKERS] SQL/MED - file_fdw

2010-11-25 Thread Shigeru HANADA
ough. Any comments and questions are welcome. Regards, -- Shigeru Hanada file_fdw.patch.gz Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] SQL/MED - postgresql_fdw

2010-11-25 Thread Shigeru HANADA
though the contents might not be enough. Any comments and questions are welcome. Regards, -- Shigeru Hanada pgsql_fdw.patch.gz Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

  1   2   3   >