Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-12-12 Thread Shigeru Hanada
(2011/12/12 22:59), Robert Haas wrote: > It does seem like this might not be enough information for the FDW to > make good decisions about pushdown. Even supposing the server on the > other hand is also PostgreSQL, the collation names might not match > (if, say, one is running Windows, and the oth

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-12-12 Thread Robert Haas
On Wed, Dec 7, 2011 at 2:34 AM, Shigeru Hanada wrote: > Sorry for delayed response. > > 2011/11/29 Albe Laurenz : >> I think that this is not always safe even from PostgreSQL to PostgreSQL. >> If two databases have different collation, "<" on strings will behave >> differently. > > Indeed.  I thin

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-12-09 Thread Greg Smith
On 12/07/2011 02:34 AM, Shigeru Hanada wrote: I think that only the owner of foreign table can keep collation consistent between foreign and local, like data type of column. We need to support per-column-collation on foreign tables too, or should deny pushing down condition which is collation-se

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-12-06 Thread Shigeru Hanada
Sorry for delayed response. 2011/11/29 Albe Laurenz : > I think that this is not always safe even from PostgreSQL to PostgreSQL. > If two databases have different collation, "<" on strings will behave > differently. Indeed.  I think that only the owner of foreign table can keep collation consiste

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-11-29 Thread Albe Laurenz
Robert Haas wrote: > 2011/11/28 Shigeru Hanada : >> I agree that allowing users to control which function/operator should be >> pushed down is useful, but GUC seems too large as unit of switching >> behavior.  "Routine Mapping", a mechanism which is defined in SQL/MED >> standard, would be the answ

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-11-28 Thread Etsuro Fujita
(2011/11/28 20:50), Shigeru Hanada wrote: > (2011/11/25 17:27), Etsuro Fujita wrote: >>So, I think it might be better to estimate >> such costs by pgsql_fdw itself without EXPLAINing on the assumption that >> a remote postgres server has the same abilities for query

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-11-28 Thread Robert Haas
2011/11/28 Shigeru Hanada : > I agree that allowing users to control which function/operator should be > pushed down is useful, but GUC seems too large as unit of switching > behavior.  "Routine Mapping", a mechanism which is defined in SQL/MED > standard, would be the answer for this issue.  It ca

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-11-28 Thread Shigeru Hanada
Hi Fujita-san, (2011/11/25 17:27), Etsuro Fujita wrote: > I'm still under reviewing, so the following is not all. I'm sorry. > estimate_costs() have been implemented to ask a remote postgres server > for the result of EXPLAIN for a remote query to get its costs such as > startup_cost and total_co

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-11-28 Thread Shigeru Hanada
Hi Kaigai-san, (2011/11/20 2:42), Kohei KaiGai wrote: > I'm still under reviewing of your patch, so the comment is not overall, sorry. Thanks for the review! > I'm not sure whether the logic of is_foreign_expr() is appropriate. > It checks oid of the function within FuncExpr and OpExpr to disall

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-11-25 Thread Etsuro Fujita
Hi Hanada-san, (2011/11/16 1:55), Shigeru Hanada wrote: > Attached are revised version of pgsql_fdw patches. I'm still under reviewing, so the following is not all. I'm sorry. estimate_costs() have been implemented to ask a remote postgres server for the result of EXPLAIN for a remote query to g

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-11-19 Thread Kohei KaiGai
Hanada-san, I'm still under reviewing of your patch, so the comment is not overall, sorry. I'm not sure whether the logic of is_foreign_expr() is appropriate. It checks oid of the function within FuncExpr and OpExpr to disallow to push down user-defined functions. However, if a user-defined opera

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-31 Thread Pavel Stehule
2011/10/31 Shigeru Hanada : > (2011/10/30 11:34), Shigeru Hanada wrote: >> 2011/10/30 Tom Lane: >>> I think we have to.  Even if we estimate that a given scan will return >>> only a few rows, what happens if we're wrong?  We don't want to blow out >>> memory on the local server by retrieving gigaby

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-29 Thread Shigeru Hanada
2011/10/30 Tom Lane : > Hitoshi Harada writes: >> On Sat, Oct 29, 2011 at 8:13 AM, Tom Lane wrote: >>> I have not looked at the code, but ISTM the way that this has to work is >>> that you set up a portal for each active scan.  Then you can fetch a few >>> rows at a time from any one of them. > >

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-29 Thread Shigeru Hanada
2011/10/29 Hitoshi Harada : > I have a doubt here, on sharing connection for each server. What if > there are simultaneous scan on the same plan? Say, > > -> Nested Loop >  -> Foreign Scan to table T1 on server A >  -> Foreign Scan to table T2 on server A > > Okay, you are thinking about Foreign Jo

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-29 Thread Tom Lane
Hitoshi Harada writes: > On Sat, Oct 29, 2011 at 8:13 AM, Tom Lane wrote: >> I have not looked at the code, but ISTM the way that this has to work is >> that you set up a portal for each active scan. Then you can fetch a few >> rows at a time from any one of them. > Hmm, true. Looking back at t

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-29 Thread Hitoshi Harada
On Sat, Oct 29, 2011 at 8:13 AM, Tom Lane wrote: > Hitoshi Harada writes: >> I have a doubt here, on sharing connection for each server. What if >> there are simultaneous scan on the same plan? Say, > >> -> Nested Loop >>   -> Foreign Scan to table T1 on server A >>   -> Foreign Scan to table T2

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-29 Thread Tom Lane
Hitoshi Harada writes: > I have a doubt here, on sharing connection for each server. What if > there are simultaneous scan on the same plan? Say, > -> Nested Loop > -> Foreign Scan to table T1 on server A > -> Foreign Scan to table T2 on server A > Okay, you are thinking about Foreign Join,

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-29 Thread Martijn van Oosterhout
On Sat, Oct 29, 2011 at 12:25:46AM -0700, Hitoshi Harada wrote: > I have a doubt here, on sharing connection for each server. What if > there are simultaneous scan on the same plan? Say, > > -> Nested Loop > -> Foreign Scan to table T1 on server A > -> Foreign Scan to table T2 on server A > >

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-29 Thread Hitoshi Harada
2011/10/25 Shigeru Hanada : > > Connection management > = > The pgsql_fdw establishes a new connection when a foreign server is > accessed first for the local session.  Established connection is shared > between all foreign scans in the local query, and shared between even > sca

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-26 Thread Tom Lane
Magnus Hagander writes: > On Wed, Oct 26, 2011 at 16:37, Tom Lane wrote: >> If that was what he meant, I'd vote against it.  There are way too many >> people who will *not* want their databases configured to be able to >> reach out onto the net.  This feature should be something that has to be >>

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-26 Thread Magnus Hagander
On Wed, Oct 26, 2011 at 19:25, Andrew Dunstan wrote: > > On 10/26/2011 12:47 PM, Magnus Hagander wrote: >>> >>> If that was what he meant, I'd vote against it.  There are way too many >>> people who will *not* want their databases configured to be able to >>> reach out onto the net.  This feature

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-26 Thread Andrew Dunstan
On 10/26/2011 12:47 PM, Magnus Hagander wrote: If that was what he meant, I'd vote against it. There are way too many people who will *not* want their databases configured to be able to reach out onto the net. This feature should be something that has to be installed by explicit user action.

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-26 Thread Magnus Hagander
On Wed, Oct 26, 2011 at 16:37, Tom Lane wrote: > Shigeru Hanada writes: >> (2011/10/25 19:15), Magnus Hagander wrote: >>> I have not looked at the code itself, but I wonder if we shouldn't >>> consider making this a part of core-proper, not just a contrib module. >>> The fact that it isn't *alrea

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-26 Thread Kohei KaiGai
2011/10/26 Robert Haas : > 2011/10/26 Shigeru Hanada : >> (2011/10/25 19:15), Magnus Hagander wrote: >>> 2011/10/25 Shigeru Hanada: I'd like to propose pgsql_fdw, FDW for external PostgreSQL server, as a contrib module.  I think that this module would be the basis of further SQL/MED

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-26 Thread Dimitri Fontaine
Stephen Frost writes: > I'm in favor of making that distinction. I would still have pgsql_fdw, > file_fdw, etc, be packaged more-or-less the same way and still use the > CREATE EXTENTION framework, of course. We called that idea “core extension” at the latest hackers meeting, and Greg Smith had

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-26 Thread Robert Haas
2011/10/26 Shigeru Hanada : > (2011/10/25 19:15), Magnus Hagander wrote: >> 2011/10/25 Shigeru Hanada: >>> I'd like to propose pgsql_fdw, FDW for external PostgreSQL server, as a >>> contrib module.  I think that this module would be the basis of further >>> SQL/MED development for core, e.g. join-

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-26 Thread Tom Lane
Shigeru Hanada writes: > (2011/10/25 19:15), Magnus Hagander wrote: >> I have not looked at the code itself, but I wonder if we shouldn't >> consider making this a part of core-proper, not just a contrib module. >> The fact that it isn't *already* available in core surprises a lot of >> people...

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-26 Thread Shigeru Hanada
(2011/10/25 19:15), Magnus Hagander wrote: > 2011/10/25 Shigeru Hanada: >> I'd like to propose pgsql_fdw, FDW for external PostgreSQL server, as a >> contrib module. I think that this module would be the basis of further >> SQL/MED development for core, e.g. join-push-down and ANALYZE support. >

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-25 Thread Marko Kreen
On Tue, Oct 25, 2011 at 3:08 PM, Tom Lane wrote: > Magnus Hagander writes: >> 2011/10/25 Shigeru Hanada : >>> I'd like to propose pgsql_fdw, FDW for external PostgreSQL server, as a >>> contrib module.  I think that this module would be the basis of further >>> SQL/MED development for core, e.g.

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-25 Thread Stephen Frost
* Kohei KaiGai (kai...@kaigai.gr.jp) wrote: > Right now, file_fdw is the only FDW module that we have in the core, Erm, guess I'm a bit confused why we've got that in core while not putting pgsql_fdw in core. This all gets back to previous discussions around 'recommended' contrib modules (which s

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-25 Thread Kohei KaiGai
>> ATM I'm not sure it's even a good idea to push pgsql_fdw into contrib. >> Once we do that its release schedule will get locked to core's --- >> wouldn't it be better to keep flexibility for now, while it's in such >> active development? > > I would be happy to keep it outside, and integrate it i

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-25 Thread Magnus Hagander
On Tue, Oct 25, 2011 at 14:08, Tom Lane wrote: > Magnus Hagander writes: >> 2011/10/25 Shigeru Hanada : >>> I'd like to propose pgsql_fdw, FDW for external PostgreSQL server, as a >>> contrib module.  I think that this module would be the basis of further >>> SQL/MED development for core, e.g. jo

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-25 Thread Tom Lane
Magnus Hagander writes: > 2011/10/25 Shigeru Hanada : >> I'd like to propose pgsql_fdw, FDW for external PostgreSQL server, as a >> contrib module.  I think that this module would be the basis of further >> SQL/MED development for core, e.g. join-push-down and ANALYZE support. > I have not looked

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-25 Thread Magnus Hagander
2011/10/25 Shigeru Hanada : > I'd like to propose pgsql_fdw, FDW for external PostgreSQL server, as a > contrib module.  I think that this module would be the basis of further > SQL/MED development for core, e.g. join-push-down and ANALYZE support. I have not looked at the code itself, but I wonde

<    1   2