Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-12-12 Thread Tom Lane
Shigeru Hanada shigeru.han...@gmail.com writes: (2011/12/12 22:59), Robert Haas wrote: ... I feel like we might need a system here that allows for more explicit user control about what to push down vs. not, rather than assuming we'll be able to figure it out behind the scenes. Agreed. How

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-12-09 Thread Greg Smith
On 12/07/2011 02:34 AM, Shigeru Hanada wrote: I think that only the owner of foreign table can keep collation consistent between foreign and local, like data type of column. We need to support per-column-collation on foreign tables too, or should deny pushing down condition which is

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-12-06 Thread Shigeru Hanada
Sorry for delayed response. 2011/11/29 Albe Laurenz laurenz.a...@wien.gv.at: I think that this is not always safe even from PostgreSQL to PostgreSQL. If two databases have different collation, on strings will behave differently. Indeed.  I think that only the owner of foreign table can keep

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-11-29 Thread Albe Laurenz
Robert Haas wrote: 2011/11/28 Shigeru Hanada shigeru.han...@gmail.com: I agree that allowing users to control which function/operator should be pushed down is useful, but GUC seems too large as unit of switching behavior.  Routine Mapping, a mechanism which is defined in SQL/MED standard,

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-11-28 Thread Shigeru Hanada
Hi Kaigai-san, (2011/11/20 2:42), Kohei KaiGai wrote: I'm still under reviewing of your patch, so the comment is not overall, sorry. Thanks for the review! I'm not sure whether the logic of is_foreign_expr() is appropriate. It checks oid of the function within FuncExpr and OpExpr to disallow

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-11-28 Thread Shigeru Hanada
Hi Fujita-san, (2011/11/25 17:27), Etsuro Fujita wrote: I'm still under reviewing, so the following is not all. I'm sorry. estimate_costs() have been implemented to ask a remote postgres server for the result of EXPLAIN for a remote query to get its costs such as startup_cost and total_cost.

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-11-28 Thread Robert Haas
2011/11/28 Shigeru Hanada shigeru.han...@gmail.com: I agree that allowing users to control which function/operator should be pushed down is useful, but GUC seems too large as unit of switching behavior.  Routine Mapping, a mechanism which is defined in SQL/MED standard, would be the answer for

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-11-28 Thread Etsuro Fujita
(2011/11/28 20:50), Shigeru Hanada wrote: (2011/11/25 17:27), Etsuro Fujita wrote: So, I think it might be better to estimate such costs by pgsql_fdw itself without EXPLAINing on the assumption that a remote postgres server has the same abilities for query

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-11-25 Thread Etsuro Fujita
Hi Hanada-san, (2011/11/16 1:55), Shigeru Hanada wrote: Attached are revised version of pgsql_fdw patches. I'm still under reviewing, so the following is not all. I'm sorry. estimate_costs() have been implemented to ask a remote postgres server for the result of EXPLAIN for a remote query to

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-11-19 Thread Kohei KaiGai
Hanada-san, I'm still under reviewing of your patch, so the comment is not overall, sorry. I'm not sure whether the logic of is_foreign_expr() is appropriate. It checks oid of the function within FuncExpr and OpExpr to disallow to push down user-defined functions. However, if a user-defined

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-31 Thread Pavel Stehule
2011/10/31 Shigeru Hanada shigeru.han...@gmail.com: (2011/10/30 11:34), Shigeru Hanada wrote: 2011/10/30 Tom Lanet...@sss.pgh.pa.us: I think we have to.  Even if we estimate that a given scan will return only a few rows, what happens if we're wrong?  We don't want to blow out memory on the

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-29 Thread Hitoshi Harada
2011/10/25 Shigeru Hanada shigeru.han...@gmail.com: Connection management = The pgsql_fdw establishes a new connection when a foreign server is accessed first for the local session.  Established connection is shared between all foreign scans in the local query, and shared

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-29 Thread Martijn van Oosterhout
On Sat, Oct 29, 2011 at 12:25:46AM -0700, Hitoshi Harada wrote: I have a doubt here, on sharing connection for each server. What if there are simultaneous scan on the same plan? Say, - Nested Loop - Foreign Scan to table T1 on server A - Foreign Scan to table T2 on server A Okay, you

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-29 Thread Tom Lane
Hitoshi Harada umi.tan...@gmail.com writes: I have a doubt here, on sharing connection for each server. What if there are simultaneous scan on the same plan? Say, - Nested Loop - Foreign Scan to table T1 on server A - Foreign Scan to table T2 on server A Okay, you are thinking about

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-29 Thread Hitoshi Harada
On Sat, Oct 29, 2011 at 8:13 AM, Tom Lane t...@sss.pgh.pa.us wrote: Hitoshi Harada umi.tan...@gmail.com writes: I have a doubt here, on sharing connection for each server. What if there are simultaneous scan on the same plan? Say, - Nested Loop   - Foreign Scan to table T1 on server A   -

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-29 Thread Tom Lane
Hitoshi Harada umi.tan...@gmail.com writes: On Sat, Oct 29, 2011 at 8:13 AM, Tom Lane t...@sss.pgh.pa.us wrote: I have not looked at the code, but ISTM the way that this has to work is that you set up a portal for each active scan. Then you can fetch a few rows at a time from any one of them.

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-29 Thread Shigeru Hanada
2011/10/29 Hitoshi Harada umi.tan...@gmail.com: I have a doubt here, on sharing connection for each server. What if there are simultaneous scan on the same plan? Say, - Nested Loop  - Foreign Scan to table T1 on server A  - Foreign Scan to table T2 on server A Okay, you are thinking about

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-29 Thread Shigeru Hanada
2011/10/30 Tom Lane t...@sss.pgh.pa.us: Hitoshi Harada umi.tan...@gmail.com writes: On Sat, Oct 29, 2011 at 8:13 AM, Tom Lane t...@sss.pgh.pa.us wrote: I have not looked at the code, but ISTM the way that this has to work is that you set up a portal for each active scan.  Then you can fetch a

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-26 Thread Shigeru Hanada
(2011/10/25 19:15), Magnus Hagander wrote: 2011/10/25 Shigeru Hanadashigeru.han...@gmail.com: I'd like to propose pgsql_fdw, FDW for external PostgreSQL server, as a contrib module. I think that this module would be the basis of further SQL/MED development for core, e.g. join-push-down and

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-26 Thread Tom Lane
Shigeru Hanada shigeru.han...@gmail.com writes: (2011/10/25 19:15), Magnus Hagander wrote: I have not looked at the code itself, but I wonder if we shouldn't consider making this a part of core-proper, not just a contrib module. The fact that it isn't *already* available in core surprises a

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-26 Thread Robert Haas
2011/10/26 Shigeru Hanada shigeru.han...@gmail.com: (2011/10/25 19:15), Magnus Hagander wrote: 2011/10/25 Shigeru Hanadashigeru.han...@gmail.com: I'd like to propose pgsql_fdw, FDW for external PostgreSQL server, as a contrib module.  I think that this module would be the basis of further

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-26 Thread Dimitri Fontaine
Stephen Frost sfr...@snowman.net writes: I'm in favor of making that distinction. I would still have pgsql_fdw, file_fdw, etc, be packaged more-or-less the same way and still use the CREATE EXTENTION framework, of course. We called that idea “core extension” at the latest hackers meeting, and

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-26 Thread Kohei KaiGai
2011/10/26 Robert Haas robertmh...@gmail.com: 2011/10/26 Shigeru Hanada shigeru.han...@gmail.com: (2011/10/25 19:15), Magnus Hagander wrote: 2011/10/25 Shigeru Hanadashigeru.han...@gmail.com: I'd like to propose pgsql_fdw, FDW for external PostgreSQL server, as a contrib module.  I think that

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-26 Thread Magnus Hagander
On Wed, Oct 26, 2011 at 16:37, Tom Lane t...@sss.pgh.pa.us wrote: Shigeru Hanada shigeru.han...@gmail.com writes: (2011/10/25 19:15), Magnus Hagander wrote: I have not looked at the code itself, but I wonder if we shouldn't consider making this a part of core-proper, not just a contrib module.

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-26 Thread Andrew Dunstan
On 10/26/2011 12:47 PM, Magnus Hagander wrote: If that was what he meant, I'd vote against it. There are way too many people who will *not* want their databases configured to be able to reach out onto the net. This feature should be something that has to be installed by explicit user

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-26 Thread Magnus Hagander
On Wed, Oct 26, 2011 at 19:25, Andrew Dunstan and...@dunslane.net wrote: On 10/26/2011 12:47 PM, Magnus Hagander wrote: If that was what he meant, I'd vote against it.  There are way too many people who will *not* want their databases configured to be able to reach out onto the net.  This

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-26 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: On Wed, Oct 26, 2011 at 16:37, Tom Lane t...@sss.pgh.pa.us wrote: If that was what he meant, I'd vote against it.  There are way too many people who will *not* want their databases configured to be able to reach out onto the net.  This feature should

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-25 Thread Magnus Hagander
2011/10/25 Shigeru Hanada shigeru.han...@gmail.com: I'd like to propose pgsql_fdw, FDW for external PostgreSQL server, as a contrib module.  I think that this module would be the basis of further SQL/MED development for core, e.g. join-push-down and ANALYZE support. I have not looked at the

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-25 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: 2011/10/25 Shigeru Hanada shigeru.han...@gmail.com: I'd like to propose pgsql_fdw, FDW for external PostgreSQL server, as a contrib module.  I think that this module would be the basis of further SQL/MED development for core, e.g. join-push-down and

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-25 Thread Magnus Hagander
On Tue, Oct 25, 2011 at 14:08, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: 2011/10/25 Shigeru Hanada shigeru.han...@gmail.com: I'd like to propose pgsql_fdw, FDW for external PostgreSQL server, as a contrib module.  I think that this module would be the basis

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-25 Thread Kohei KaiGai
ATM I'm not sure it's even a good idea to push pgsql_fdw into contrib. Once we do that its release schedule will get locked to core's --- wouldn't it be better to keep flexibility for now, while it's in such active development? I would be happy to keep it outside, and integrate it in the

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-25 Thread Stephen Frost
* Kohei KaiGai (kai...@kaigai.gr.jp) wrote: Right now, file_fdw is the only FDW module that we have in the core, Erm, guess I'm a bit confused why we've got that in core while not putting pgsql_fdw in core. This all gets back to previous discussions around 'recommended' contrib modules (which

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2011-10-25 Thread Marko Kreen
On Tue, Oct 25, 2011 at 3:08 PM, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: 2011/10/25 Shigeru Hanada shigeru.han...@gmail.com: I'd like to propose pgsql_fdw, FDW for external PostgreSQL server, as a contrib module.  I think that this module would be the

<    1   2