Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-04-02 Thread Fujii Masao
On 2021/04/02 2:22, Fujii Masao wrote: Thanks a lot! Barring any objection, I will commit this version. Pushed. Thanks! Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-04-01 Thread Fujii Masao
On 2021/04/02 1:13, Bharath Rupireddy wrote: On Thu, Apr 1, 2021 at 8:56 PM Fujii Masao wrote: Attaching v21 patch set, rebased onto the latest master. I agree to add the server-level option. But I'm still not sure if it's good idea to also expose that option as GUC. Isn't the

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-04-01 Thread Bharath Rupireddy
On Thu, Apr 1, 2021 at 8:56 PM Fujii Masao wrote: > > > Attaching v21 patch set, rebased onto the latest master. > > I agree to add the server-level option. But I'm still not sure if it's good > idea to also expose that option as GUC. Isn't the server-level option enough > for most cases? > >

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-04-01 Thread Fujii Masao
On 2021/02/22 14:55, Bharath Rupireddy wrote: On Thu, Feb 4, 2021 at 9:36 AM Bharath Rupireddy wrote: On Wed, Feb 3, 2021 at 4:22 PM Fujii Masao wrote: Maybe my explanation in the previous email was unclear. What I think is; If the server-level option is explicitly specified, its setting

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-02-21 Thread Bharath Rupireddy
On Thu, Feb 4, 2021 at 9:36 AM Bharath Rupireddy wrote: > > On Wed, Feb 3, 2021 at 4:22 PM Fujii Masao > wrote: > > Maybe my explanation in the previous email was unclear. What I think is; If > > the server-level option is explicitly specified, its setting is used > > whatever GUC is. On the

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-02-03 Thread Bharath Rupireddy
On Wed, Feb 3, 2021 at 4:22 PM Fujii Masao wrote: > Maybe my explanation in the previous email was unclear. What I think is; If > the server-level option is explicitly specified, its setting is used whatever > GUC is. On the other hand, if the server-level option is NOT specified, GUC >

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-02-03 Thread Fujii Masao
On 2021/02/03 13:56, Bharath Rupireddy wrote: On Tue, Feb 2, 2021 at 9:45 AM Fujii Masao wrote: One merit of keep_connections that I found is that we can use it even when connecting to the older PostgreSQL that doesn't support idle_session_timeout. Also it seems simpler to use

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-02-02 Thread Bharath Rupireddy
On Tue, Feb 2, 2021 at 9:45 AM Fujii Masao wrote: > One merit of keep_connections that I found is that we can use it even > when connecting to the older PostgreSQL that doesn't support > idle_session_timeout. Also it seems simpler to use keep_connections > rather than setting idle_session_timeout

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-02-01 Thread Fujii Masao
On 2021/02/01 16:39, Bharath Rupireddy wrote: On Mon, Feb 1, 2021 at 12:43 PM Fujii Masao wrote: On 2021/01/27 10:06, Bharath Rupireddy wrote: On Tue, Jan 26, 2021 at 8:38 AM Bharath Rupireddy wrote: I will post "keep_connections" GUC and "keep_connection" server level option patches

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-02-01 Thread Fujii Masao
On 2021/02/01 16:13, Bharath Rupireddy wrote: On Mon, Feb 1, 2021 at 12:29 PM Fujii Masao wrote: On 2021/01/30 9:28, Bharath Rupireddy wrote: On Sat, Jan 30, 2021 at 12:14 AM Fujii Masao wrote: + /* +* It doesn't make sense to show this entry

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-31 Thread Bharath Rupireddy
On Mon, Feb 1, 2021 at 12:43 PM Fujii Masao wrote: > On 2021/01/27 10:06, Bharath Rupireddy wrote: > > On Tue, Jan 26, 2021 at 8:38 AM Bharath Rupireddy > > wrote: > >> I will post "keep_connections" GUC and "keep_connection" server level > >> option patches later. > > > > Attaching v19 patch

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-31 Thread Bharath Rupireddy
On Mon, Feb 1, 2021 at 12:29 PM Fujii Masao wrote: > On 2021/01/30 9:28, Bharath Rupireddy wrote: > > On Sat, Jan 30, 2021 at 12:14 AM Fujii Masao > > wrote: > >> + /* > >> +* It doesn't make sense to show this entry in the > >> output with a > >> +

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-31 Thread Fujii Masao
On 2021/01/27 10:06, Bharath Rupireddy wrote: On Tue, Jan 26, 2021 at 8:38 AM Bharath Rupireddy wrote: I will post "keep_connections" GUC and "keep_connection" server level option patches later. Attaching v19 patch set for "keep_connections" GUC and "keep_connection" server level option.

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-31 Thread Fujii Masao
On 2021/01/30 9:28, Bharath Rupireddy wrote: On Sat, Jan 30, 2021 at 12:14 AM Fujii Masao wrote: + /* +* It doesn't make sense to show this entry in the output with a +* NULL server_name as it will be closed at the xact

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-29 Thread Bharath Rupireddy
On Sat, Jan 30, 2021 at 12:14 AM Fujii Masao wrote: > + /* > +* It doesn't make sense to show this entry in the > output with a > +* NULL server_name as it will be closed at the xact > end. > +*/ > +

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-29 Thread Fujii Masao
On 2021/01/29 19:45, Bharath Rupireddy wrote: On Fri, Jan 29, 2021 at 1:24 PM Bharath Rupireddy wrote: On Fri, Jan 29, 2021 at 1:17 PM Fujii Masao wrote: But if the issue is only the inconsistency of test results, we can go with the option (2)? Even with (2), we can make the test stable

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-29 Thread Bharath Rupireddy
On Fri, Jan 29, 2021 at 1:24 PM Bharath Rupireddy wrote: > > On Fri, Jan 29, 2021 at 1:17 PM Fujii Masao > wrote: > > >> But if the issue is only the inconsistency of test results, > > >> we can go with the option (2)? Even with (2), we can make the test > > >> stable by removing "valid" column

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-28 Thread Bharath Rupireddy
On Fri, Jan 29, 2021 at 1:17 PM Fujii Masao wrote: > >> But if the issue is only the inconsistency of test results, > >> we can go with the option (2)? Even with (2), we can make the test > >> stable by removing "valid" column and executing > >> postgres_fdw_get_connections() within the

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-28 Thread Fujii Masao
On 2021/01/29 16:12, Bharath Rupireddy wrote: On Fri, Jan 29, 2021 at 12:36 PM Fujii Masao wrote: On 2021/01/29 15:44, Bharath Rupireddy wrote: On Fri, Jan 29, 2021 at 11:54 AM Fujii Masao wrote: IIRC, when we were finding a way to close the invalidated connections so that they don't

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-28 Thread Bharath Rupireddy
On Fri, Jan 29, 2021 at 12:36 PM Fujii Masao wrote: > On 2021/01/29 15:44, Bharath Rupireddy wrote: > > On Fri, Jan 29, 2021 at 11:54 AM Fujii Masao > > wrote: > IIRC, when we were finding a way to close the invalidated connections > so that they don't leaked, we had two options: >

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-28 Thread Fujii Masao
On 2021/01/29 15:44, Bharath Rupireddy wrote: On Fri, Jan 29, 2021 at 11:54 AM Fujii Masao wrote: IIRC, when we were finding a way to close the invalidated connections so that they don't leaked, we had two options: 1) let those connections (whether currently being used in the xact or not)

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-28 Thread Bharath Rupireddy
On Fri, Jan 29, 2021 at 11:54 AM Fujii Masao wrote: > >> IIRC, when we were finding a way to close the invalidated connections > >> so that they don't leaked, we had two options: > >> > >> 1) let those connections (whether currently being used in the xact or > >> not) get marked invalidated in

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-28 Thread Fujii Masao
On 2021/01/29 14:46, Bharath Rupireddy wrote: On Fri, Jan 29, 2021 at 11:08 AM Bharath Rupireddy wrote: On Fri, Jan 29, 2021 at 10:55 AM Fujii Masao wrote: On 2021/01/29 14:12, Bharath Rupireddy wrote: On Fri, Jan 29, 2021 at 10:28 AM Fujii Masao wrote: On 2021/01/29 11:09, Tom Lane

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-28 Thread Bharath Rupireddy
On Fri, Jan 29, 2021 at 11:38 AM Fujii Masao wrote: > On 2021/01/29 14:53, Bharath Rupireddy wrote: > > On Fri, Jan 29, 2021 at 10:55 AM Fujii Masao > > wrote: > BTW, even if we change pgfdw_inval_callback() so that it doesn't close > the connection at all, ISTM that the results of >

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-28 Thread Fujii Masao
On 2021/01/29 14:53, Bharath Rupireddy wrote: On Fri, Jan 29, 2021 at 10:55 AM Fujii Masao wrote: BTW, even if we change pgfdw_inval_callback() so that it doesn't close the connection at all, ISTM that the results of postgres_fdw_get_connections() would not be stable because

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-28 Thread Bharath Rupireddy
On Fri, Jan 29, 2021 at 10:55 AM Fujii Masao wrote: > >> BTW, even if we change pgfdw_inval_callback() so that it doesn't close > >> the connection at all, ISTM that the results of > >> postgres_fdw_get_connections() > >> would not be stable because entry->invalidated would vary based on > >>

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-28 Thread Bharath Rupireddy
On Fri, Jan 29, 2021 at 11:08 AM Bharath Rupireddy wrote: > > On Fri, Jan 29, 2021 at 10:55 AM Fujii Masao > wrote: > > On 2021/01/29 14:12, Bharath Rupireddy wrote: > > > On Fri, Jan 29, 2021 at 10:28 AM Fujii Masao > > > wrote: > > >> On 2021/01/29 11:09, Tom Lane wrote: > > >>> Bharath

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-28 Thread Bharath Rupireddy
On Fri, Jan 29, 2021 at 10:55 AM Fujii Masao wrote: > On 2021/01/29 14:12, Bharath Rupireddy wrote: > > On Fri, Jan 29, 2021 at 10:28 AM Fujii Masao > > wrote: > >> On 2021/01/29 11:09, Tom Lane wrote: > >>> Bharath Rupireddy writes: > On Fri, Jan 29, 2021 at 1:52 AM Tom Lane wrote: >

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-28 Thread Fujii Masao
On 2021/01/29 14:12, Bharath Rupireddy wrote: On Fri, Jan 29, 2021 at 10:28 AM Fujii Masao wrote: On 2021/01/29 11:09, Tom Lane wrote: Bharath Rupireddy writes: On Fri, Jan 29, 2021 at 1:52 AM Tom Lane wrote:

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-28 Thread Bharath Rupireddy
On Fri, Jan 29, 2021 at 10:42 AM Bharath Rupireddy wrote: > > > Also, now that I've looked at pgfdw_inval_callback, it scares > > > the heck out of me. Actually disconnecting a connection during > > > a cache inval callback seems quite unsafe --- what if that happens > > > while we're using the

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-28 Thread Bharath Rupireddy
On Fri, Jan 29, 2021 at 10:28 AM Fujii Masao wrote: > On 2021/01/29 11:09, Tom Lane wrote: > > Bharath Rupireddy writes: > >> On Fri, Jan 29, 2021 at 1:52 AM Tom Lane wrote: > >>> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=trilobite=2021-01-26%2019%3A59%3A40 > >>> This is a

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-28 Thread Fujii Masao
On 2021/01/29 11:09, Tom Lane wrote: Bharath Rupireddy writes: On Fri, Jan 29, 2021 at 1:52 AM Tom Lane wrote: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=trilobite=2021-01-26%2019%3A59%3A40 This is a CLOBBER_CACHE_ALWAYS build, so I suspect what it's telling us is that the

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-28 Thread Tom Lane
Bharath Rupireddy writes: > On Fri, Jan 29, 2021 at 1:52 AM Tom Lane wrote: >> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=trilobite=2021-01-26%2019%3A59%3A40 >> This is a CLOBBER_CACHE_ALWAYS build, so I suspect what it's >> telling us is that the patch's behavior is unstable in the

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-28 Thread Bharath Rupireddy
On Fri, Jan 29, 2021 at 1:52 AM Tom Lane wrote: > > Bharath Rupireddy writes: > > On Tue, Jan 26, 2021 at 1:55 PM Fujii Masao > > wrote: > >> Thanks for the patch! I also created that patch, confirmed that the test > >> successfully passed with -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS, > >>

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-28 Thread Tom Lane
Bharath Rupireddy writes: > On Tue, Jan 26, 2021 at 1:55 PM Fujii Masao > wrote: >> Thanks for the patch! I also created that patch, confirmed that the test >> successfully passed with -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS, >> and pushed the patch. > Thanks a lot! Seems you're not out

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-26 Thread Bharath Rupireddy
On Tue, Jan 26, 2021 at 8:38 AM Bharath Rupireddy wrote: > I will post "keep_connections" GUC and "keep_connection" server level > option patches later. Attaching v19 patch set for "keep_connections" GUC and "keep_connection" server level option. Please review them further. With Regards,

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-26 Thread Bharath Rupireddy
On Tue, Jan 26, 2021 at 1:55 PM Fujii Masao wrote: > Thanks for the patch! I also created that patch, confirmed that the test > successfully passed with -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS, > and pushed the patch. Thanks a lot! With Regards, Bharath Rupireddy. EnterpriseDB:

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-26 Thread Fujii Masao
On 2021/01/26 17:07, Bharath Rupireddy wrote: On Tue, Jan 26, 2021 at 1:27 PM Fujii Masao wrote: Yes, so I pushed that change to stabilize the regression test. Let's keep checking how the results of buildfarm members are changed. Sorry, I'm unfamiliar with checking the system status on

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-26 Thread Bharath Rupireddy
On Tue, Jan 26, 2021 at 1:27 PM Fujii Masao wrote: > > Yes, so I pushed that change to stabilize the regression test. > > Let's keep checking how the results of buildfarm members are changed. Sorry, I'm unfamiliar with checking the system status on the build farm website -

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-25 Thread Fujii Masao
On 2021/01/26 16:39, Fujii Masao wrote: On 2021/01/26 16:33, Bharath Rupireddy wrote: On Tue, Jan 26, 2021 at 12:54 PM Fujii Masao wrote: On 2021/01/26 16:05, Tom Lane wrote: Fujii Masao writes: Thanks for the review! I fixed them and pushed the patch! Buildfarm is very not happy

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-25 Thread Fujii Masao
On 2021/01/26 16:33, Bharath Rupireddy wrote: On Tue, Jan 26, 2021 at 12:54 PM Fujii Masao wrote: On 2021/01/26 16:05, Tom Lane wrote: Fujii Masao writes: Thanks for the review! I fixed them and pushed the patch! Buildfarm is very not happy ... Yes I'm investigating that. --

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-25 Thread Bharath Rupireddy
On Tue, Jan 26, 2021 at 12:54 PM Fujii Masao wrote: > On 2021/01/26 16:05, Tom Lane wrote: > > Fujii Masao writes: > >> Thanks for the review! I fixed them and pushed the patch! > > > > Buildfarm is very not happy ... > > Yes I'm investigating that. > > -- Return false as connections are

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-25 Thread Fujii Masao
On 2021/01/26 16:05, Tom Lane wrote: Fujii Masao writes: Thanks for the review! I fixed them and pushed the patch! Buildfarm is very not happy ... Yes I'm investigating that. -- Return false as connections are still in use, warnings are issued. SELECT

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-25 Thread Tom Lane
Fujii Masao writes: > Thanks for the review! I fixed them and pushed the patch! Buildfarm is very not happy ... regards, tom lane

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-25 Thread Fujii Masao
On 2021/01/26 12:08, Bharath Rupireddy wrote: On Tue, Jan 26, 2021 at 12:38 AM Fujii Masao wrote: Attaching v17 patch set, please review it further. Thanks for updating the patch! Attached is the tweaked version of the patch. I didn't change any logic, but I updated some comments and

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-25 Thread Bharath Rupireddy
On Tue, Jan 26, 2021 at 12:38 AM Fujii Masao wrote: > > Attaching v17 patch set, please review it further. > > Thanks for updating the patch! > > Attached is the tweaked version of the patch. I didn't change any logic, > but I updated some comments and docs. Also I added the regresssion test > to

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-25 Thread Fujii Masao
On 2021/01/26 0:12, Bharath Rupireddy wrote: On Mon, Jan 25, 2021 at 7:28 PM Bharath Rupireddy wrote: I will provide the updated patch set soon. Attaching v17 patch set, please review it further. Thanks for updating the patch! Attached is the tweaked version of the patch. I didn't

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-25 Thread Bharath Rupireddy
On Mon, Jan 25, 2021 at 7:28 PM Bharath Rupireddy wrote: > I will provide the updated patch set soon. Attaching v17 patch set, please review it further. With Regards, Bharath Rupireddy. EnterpriseDB: http://www.enterprisedb.com v17-0001-postgres_fdw-function-to-discard-cached-connecti.patch

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-25 Thread Bharath Rupireddy
On Mon, Jan 25, 2021 at 7:20 PM Fujii Masao wrote: > On 2021/01/25 19:28, Bharath Rupireddy wrote: > > On Mon, Jan 25, 2021 at 3:17 PM Fujii Masao > > wrote: > >>> Yes, if required backends can establish the connection again. But my > >>> worry is this - a non-super user disconnecting all or a

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-25 Thread Fujii Masao
On 2021/01/25 19:28, Bharath Rupireddy wrote: On Mon, Jan 25, 2021 at 3:17 PM Fujii Masao wrote: Yes, if required backends can establish the connection again. But my worry is this - a non-super user disconnecting all or a given connection created by a super user? Yes, I was also worried

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-25 Thread Bharath Rupireddy
On Mon, Jan 25, 2021 at 3:17 PM Fujii Masao wrote: > > Yes, if required backends can establish the connection again. But my > > worry is this - a non-super user disconnecting all or a given > > connection created by a super user? > > Yes, I was also worried about that. But I found that there are

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-25 Thread Fujii Masao
On 2021/01/25 18:13, Bharath Rupireddy wrote: On Mon, Jan 25, 2021 at 1:20 PM Fujii Masao wrote: Yeah, connections can be discarded by non-super users using postgres_fdw_disconnect_all and postgres_fdw_disconnect. Given the fact that a non-super user requires a password to access foreign

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-25 Thread Bharath Rupireddy
On Mon, Jan 25, 2021 at 1:20 PM Fujii Masao wrote: > > Yeah, connections can be discarded by non-super users using > > postgres_fdw_disconnect_all and postgres_fdw_disconnect. Given the > > fact that a non-super user requires a password to access foreign > > tables [1], IMO a non-super user

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-24 Thread Fujii Masao
On 2021/01/23 13:40, Bharath Rupireddy wrote: On Fri, Jan 22, 2021 at 6:43 PM Fujii Masao wrote: Please review the v16 patch set further. Thanks! Will review that later. + /* +* For the given server, if we closed connection or it is still

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-22 Thread Bharath Rupireddy
On Fri, Jan 22, 2021 at 6:43 PM Fujii Masao wrote: > >> Please review the v16 patch set further. > > > > Thanks! Will review that later. > > + /* > +* For the given server, if we closed connection or > it is still in > +* use,

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-22 Thread Fujii Masao
On 2021/01/22 3:29, Fujii Masao wrote: On 2021/01/22 1:17, Bharath Rupireddy wrote: On Thu, Jan 21, 2021 at 8:58 PM Fujii Masao wrote: My opinion is to check "!all", but if others prefer using such boolean flag, I'd withdraw my opinion. I'm really sorry, actually if (!all) is enough

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-21 Thread Fujii Masao
On 2021/01/22 1:17, Bharath Rupireddy wrote: On Thu, Jan 21, 2021 at 8:58 PM Fujii Masao wrote: My opinion is to check "!all", but if others prefer using such boolean flag, I'd withdraw my opinion. I'm really sorry, actually if (!all) is enough there, my earlier understanding was wrong.

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-21 Thread Bharath Rupireddy
On Thu, Jan 21, 2021 at 8:58 PM Fujii Masao wrote: > My opinion is to check "!all", but if others prefer using such boolean flag, > I'd withdraw my opinion. I'm really sorry, actually if (!all) is enough there, my earlier understanding was wrong. > + if ((all ||

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-21 Thread Fujii Masao
On 2021/01/21 16:16, Bharath Rupireddy wrote: On Thu, Jan 21, 2021 at 12:17 PM Fujii Masao wrote: On 2021/01/21 14:46, Bharath Rupireddy wrote: On Thu, Jan 21, 2021 at 10:06 AM Fujii Masao wrote: > >> + if (entry->server_hashvalue == hashvalue && +

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-20 Thread Bharath Rupireddy
On Thu, Jan 21, 2021 at 12:17 PM Fujii Masao wrote: > On 2021/01/21 14:46, Bharath Rupireddy wrote: > > On Thu, Jan 21, 2021 at 10:06 AM Fujii Masao > > wrote: > > > >> + if (entry->server_hashvalue == hashvalue && > + (entry->xact_depth

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-20 Thread Fujii Masao
On 2021/01/21 14:46, Bharath Rupireddy wrote: On Thu, Jan 21, 2021 at 10:06 AM Fujii Masao wrote: > >> + if (entry->server_hashvalue == hashvalue && + (entry->xact_depth > 0 || result)) + { +

RE: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-20 Thread Hou, Zhijie
> > > Attaching v15 patch set. Please consider it for further review. > > > > Hi > > > > I have some comments for the 0001 patch > > > > In v15-0001-postgres_fdw-function-to-discard-cached-connecti > > > > 1. > > + If there is no open connection to the given foreign server, > false > > +

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-20 Thread Bharath Rupireddy
On Thu, Jan 21, 2021 at 11:15 AM Hou, Zhijie wrote: > > > Attaching v15 patch set. Please consider it for further review. > > Hi > > I have some comments for the 0001 patch > > In v15-0001-postgres_fdw-function-to-discard-cached-connecti > > 1. > + If there is no open connection to the given

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-20 Thread Bharath Rupireddy
On Thu, Jan 21, 2021 at 10:06 AM Fujii Masao wrote: > >> + if (entry->server_hashvalue == hashvalue && > >> + (entry->xact_depth > 0 || result)) > >> + { > >> + hash_seq_term(); > >> +

RE: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-20 Thread Hou, Zhijie
> Attaching v15 patch set. Please consider it for further review. Hi I have some comments for the 0001 patch In v15-0001-postgres_fdw-function-to-discard-cached-connecti 1. + If there is no open connection to the given foreign server, false + is returned. If no foreign server with

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-20 Thread Fujii Masao
On 2021/01/21 12:00, Bharath Rupireddy wrote: On Wed, Jan 20, 2021 at 6:58 PM Fujii Masao wrote: + * It checks if the cache has a connection for the given foreign server that's + * not being used within current transaction, then returns true. If the + * connection is in use, then it emits a

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-20 Thread Bharath Rupireddy
On Wed, Jan 20, 2021 at 6:58 PM Fujii Masao wrote: > + * It checks if the cache has a connection for the given foreign server > that's > + * not being used within current transaction, then returns true. If the > + * connection is in use, then it emits a warning and returns false. > > The comment

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-20 Thread Fujii Masao
On 2021/01/20 19:17, Bharath Rupireddy wrote: On Wed, Jan 20, 2021 at 3:24 PM Fujii Masao wrote: Keeping above in mind, I feel we can do hash_seq_search(), as we do currently, even when the server name is given as input. This way, we don't need to bother much on the above points. Thoughts?

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-20 Thread Bharath Rupireddy
On Wed, Jan 20, 2021 at 3:24 PM Fujii Masao wrote: > > Keeping above in mind, I feel we can do hash_seq_search(), as we do > > currently, even when the server name is given as input. This way, we > > don't need to bother much on the above points. > > > > Thoughts? > > Thanks for explaining this!

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-20 Thread Fujii Masao
On 2021/01/20 17:41, Bharath Rupireddy wrote: On Wed, Jan 20, 2021 at 11:53 AM Fujii Masao wrote: So, furthermore, we can use hash_search() to find the target cached connection, instead of using hash_seq_search(), when the server name is given. This would simplify the code a bit more? Of

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-20 Thread Bharath Rupireddy
On Wed, Jan 20, 2021 at 11:53 AM Fujii Masao wrote: > So, furthermore, we can use hash_search() to find the target cached > connection, instead of using hash_seq_search(), when the server name > is given. This would simplify the code a bit more? Of course, > hash_seq_search() is necessary when

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-19 Thread Fujii Masao
On 2021/01/19 12:09, Bharath Rupireddy wrote: On Mon, Jan 18, 2021 at 9:11 PM Fujii Masao wrote: Attaching v12 patch set. 0001 is for postgres_fdw_disconnect() function, 0002 is for keep_connections GUC and 0003 is for keep_connection server level option. Thanks! Please review it

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-18 Thread Fujii Masao
On 2021/01/19 9:53, Hou, Zhijie wrote: +1 to add it after "dropped (Note )", how about as follows with slight changes? dropped (Note that server name of an invalid connection can be NULL if the server is dropped), and then such . Yes, I like this one. One question is; "should"

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-18 Thread Bharath Rupireddy
On Mon, Jan 18, 2021 at 9:11 PM Fujii Masao wrote: > > Attaching v12 patch set. 0001 is for postgres_fdw_disconnect() > > function, 0002 is for keep_connections GUC and 0003 is for > > keep_connection server level option. > > Thanks! > > > > > Please review it further. > > + server

RE: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-18 Thread Hou, Zhijie
> >>> +1 to add it after "dropped (Note )", how about as follows > >>> with slight changes? > >>> > >>> dropped (Note that server name of an invalid connection can be NULL > >>> if the server is dropped), and then such . > >> > >> Yes, I like this one. One question is; "should" or "is"

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-18 Thread Fujii Masao
On 2021/01/18 22:03, Bharath Rupireddy wrote: On Mon, Jan 18, 2021 at 6:17 PM Fujii Masao wrote: +1 to add it after "dropped (Note )", how about as follows with slight changes? dropped (Note that server name of an invalid connection can be NULL if the server is dropped), and then

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-18 Thread Fujii Masao
On 2021/01/18 23:14, Bharath Rupireddy wrote: On Mon, Jan 18, 2021 at 11:44 AM Fujii Masao wrote: I will post patches for the other function postgres_fdw_disconnect, GUC and server level option later. Thanks! Attaching v12 patch set. 0001 is for postgres_fdw_disconnect() function, 0002

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-18 Thread Bharath Rupireddy
On Mon, Jan 18, 2021 at 11:44 AM Fujii Masao wrote: > > I will post patches for the other function postgres_fdw_disconnect, > > GUC and server level option later. > > Thanks! Attaching v12 patch set. 0001 is for postgres_fdw_disconnect() function, 0002 is for keep_connections GUC and 0003 is for

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-18 Thread Bharath Rupireddy
On Mon, Jan 18, 2021 at 6:17 PM Fujii Masao wrote: > > +1 to add it after "dropped (Note )", how about as follows > > with slight changes? > > > > dropped (Note that server name of an invalid connection can be NULL if > > the server is dropped), and then such . > > Yes, I like this

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-18 Thread Fujii Masao
On 2021/01/18 15:37, Bharath Rupireddy wrote: On Mon, Jan 18, 2021 at 11:58 AM Fujii Masao wrote: On 2021/01/18 15:02, Hou, Zhijie wrote: We need to create the loopback3 with user mapping public, otherwise the test might become unstable as shown below. Note that loopback and loopback2

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-17 Thread Bharath Rupireddy
On Mon, Jan 18, 2021 at 11:58 AM Fujii Masao wrote: > > > > On 2021/01/18 15:02, Hou, Zhijie wrote: > >> We need to create the loopback3 with user mapping public, otherwise the > >> test might become unstable as shown below. Note that loopback and > >> loopback2 are not dropped in the test, so no

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-17 Thread Fujii Masao
On 2021/01/18 15:02, Hou, Zhijie wrote: We need to create the loopback3 with user mapping public, otherwise the test might become unstable as shown below. Note that loopback and loopback2 are not dropped in the test, so no problem with them. ALTER SERVER loopback OPTIONS (ADD

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-17 Thread Fujii Masao
On 2021/01/18 13:46, Bharath Rupireddy wrote: On Mon, Jan 18, 2021 at 9:38 AM Fujii Masao wrote: Please review v11 further. Thanks for updating the patch! The patch for postgres_fdw_get_connections() basically looks good to me. So at first I'd like to push it. Attached is the patch that

RE: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-17 Thread Hou, Zhijie
> We need to create the loopback3 with user mapping public, otherwise the > test might become unstable as shown below. Note that loopback and > loopback2 are not dropped in the test, so no problem with them. > > ALTER SERVER loopback OPTIONS (ADD use_remote_estimate 'off'); DROP > SERVER

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-17 Thread Bharath Rupireddy
On Mon, Jan 18, 2021 at 9:38 AM Fujii Masao wrote: > > Please review v11 further. > > Thanks for updating the patch! > > The patch for postgres_fdw_get_connections() basically looks good to me. > So at first I'd like to push it. Attached is the patch that I extracted >

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-17 Thread Fujii Masao
On 2021/01/18 12:33, Bharath Rupireddy wrote: On Sun, Jan 17, 2021 at 11:30 PM Zhihong Yu wrote: This patch introduces new function postgres_fdw_disconnect() when called with a foreign server name discards the associated connections with the server name. I think the following would read

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-17 Thread Bharath Rupireddy
On Sun, Jan 17, 2021 at 11:30 PM Zhihong Yu wrote: > This patch introduces new function postgres_fdw_disconnect() when > called with a foreign server name discards the associated > connections with the server name. > > I think the following would read better: > > This patch introduces a new

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-17 Thread Zhihong Yu
Hi, This patch introduces new function postgres_fdw_disconnect() when called with a foreign server name discards the associated connections with the server name. I think the following would read better: This patch introduces *a* new function postgres_fdw_disconnect(). When called with a foreign

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-16 Thread Bharath Rupireddy
On Sat, Jan 16, 2021 at 10:36 AM Bharath Rupireddy wrote: > > > Please consider the v9 patch set for further review. > > > > Thanks for updating the patch! I reviewed only 0001 patch. I addressed the review comments and attached v10 patch set. Please consider it for further review. With

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-15 Thread Bharath Rupireddy
On Thu, Jan 14, 2021 at 3:52 PM Fujii Masao wrote: > On 2021/01/09 10:12, Bharath Rupireddy wrote: > > On Fri, Jan 8, 2021 at 9:55 AM Bharath Rupireddy > > wrote: > >> I will make the changes and post a new patch set soon. > > > > Attaching v9 patch set that has addressed the review comments on

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-14 Thread Fujii Masao
On 2021/01/14 20:36, Bharath Rupireddy wrote: On Thu, Jan 14, 2021 at 3:52 PM Fujii Masao wrote: - if (!HeapTupleIsValid(tup)) - elog(ERROR, "cache lookup failed for user mapping %u", entry->key); - umform = (Form_pg_user_mapping) GETSTRUCT(tup); - server =

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-14 Thread Bharath Rupireddy
On Thu, Jan 14, 2021 at 3:52 PM Fujii Masao wrote: > - if (!HeapTupleIsValid(tup)) > - elog(ERROR, "cache lookup failed for user mapping %u", > entry->key); > - umform = (Form_pg_user_mapping) GETSTRUCT(tup); > - server = GetForeignServer(umform->umserver); > -

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-14 Thread Fujii Masao
On 2021/01/09 10:12, Bharath Rupireddy wrote: On Fri, Jan 8, 2021 at 9:55 AM Bharath Rupireddy wrote: I will make the changes and post a new patch set soon. Attaching v9 patch set that has addressed the review comments on the disconnect function returning setof records, documentation

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-08 Thread Bharath Rupireddy
On Fri, Jan 8, 2021 at 9:55 AM Bharath Rupireddy wrote: > I will make the changes and post a new patch set soon. Attaching v9 patch set that has addressed the review comments on the disconnect function returning setof records, documentation changes, and postgres_fdw--1.0-1.1.sql changes. Please

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-07 Thread Bharath Rupireddy
On Fri, Jan 8, 2021 at 7:29 AM Fujii Masao wrote: > On 2021/01/07 17:21, Bharath Rupireddy wrote: > > On Thu, Jan 7, 2021 at 9:49 AM Fujii Masao > > wrote: > >> On 2021/01/05 16:56, Bharath Rupireddy wrote: > >>> Attaching v8 patch set. Hopefully, cf bot will be happy with v8. > >>> > >>>

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-07 Thread Fujii Masao
On 2021/01/07 17:21, Bharath Rupireddy wrote: On Thu, Jan 7, 2021 at 9:49 AM Fujii Masao wrote: On 2021/01/05 16:56, Bharath Rupireddy wrote: Attaching v8 patch set. Hopefully, cf bot will be happy with v8. Please consider the v8 patch set for further review. -DATA =

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-07 Thread Bharath Rupireddy
On Thu, Jan 7, 2021 at 9:49 AM Fujii Masao wrote: > On 2021/01/05 16:56, Bharath Rupireddy wrote: > > Attaching v8 patch set. Hopefully, cf bot will be happy with v8. > > > > Please consider the v8 patch set for further review. > -DATA = postgres_fdw--1.0.sql > +DATA = postgres_fdw--1.1.sql

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-06 Thread Fujii Masao
On 2021/01/05 16:56, Bharath Rupireddy wrote: On Sat, Jan 2, 2021 at 11:19 AM Bharath Rupireddy wrote: I'm sorry for the mess. I missed adding the new files into the v6-0001 patch. Please ignore the v6 patch set and consder the v7 patch set for further review. Note that 0002 and 0003

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-04 Thread Bharath Rupireddy
On Sat, Jan 2, 2021 at 11:19 AM Bharath Rupireddy wrote: > I'm sorry for the mess. I missed adding the new files into the v6-0001 > patch. Please ignore the v6 patch set and consder the v7 patch set for > further review. Note that 0002 and 0003 patches have no difference > from v5 patch set. It

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-01 Thread Bharath Rupireddy
On Sat, Jan 2, 2021 at 10:53 AM Bharath Rupireddy wrote: > Thanks for taking a look at the patches. > > On Fri, Jan 1, 2021 at 9:35 PM Zhihong Yu wrote: > > Happy new year. > > > > + appendStringInfo(, "(%s, %s)", server->servername, > > +entry->invalidated ?

  1   2   >