Re: [External] Re: FDW, too long to run explain

2019-02-18 Thread Vijaykumar Jain
Oh Wow, i guess you are right. I just ran example where local runs make use of parallel setup, but not FDW. i have three servers 2 x pg10 1 x pg11 i run queries on coordinator node ( pg11 ) which makes calls to foreign server to do a simple count. the individual nodes run the query in parallel,

Re: [External] Re: FDW, too long to run explain

2019-02-17 Thread Jeff Janes
On Sun, Feb 17, 2019 at 6:32 AM Vijaykumar Jain wrote: > I am yet to figure out the reason, what we have done is implement fake > columns to represent samples and giving them random numbers and keeping > other bulls to fake limit. > > Most of the queries that were impacted were the ones that did

Re: [External] Re: FDW, too long to run explain

2019-02-17 Thread Jeff Janes
On Sun, Feb 17, 2019 at 2:37 PM Vijaykumar Jain wrote: > > Ok, i raked this from the logs where enabled log_min_duration_statement = > 10s > > 2019-01-31 12:48:18 UTC LOG: duration: 29863.311 ms statement: EXPLAIN > SELECT blah, FROM public.view WHERE ((scheduled_bdt >= '2019-01-20'::date)) >

Re: FDW, too long to run explain

2019-02-17 Thread auxsvr
On Sunday, 17 February 2019 20:58:47 EET Jeff Janes wrote: > A question for the PostgreSQL hackers would be, Is it necessary and > desirable that the EXPLAIN be issued in the same transaction as the > eventual DECLARE and FETCHes? I don't think it is. I guess if the foreign > side table

Re: [External] Re: FDW, too long to run explain

2019-02-17 Thread Vijaykumar Jain
Regards, Vijay On Mon, Feb 18, 2019 at 12:56 AM Jeff Janes wrote: > On Sun, Feb 17, 2019 at 1:52 PM Vijaykumar Jain > wrote: > >> Assuming your questions as 1,2,3, please find my answers below. >> >> 1)"explain" on foreign servers run as "idle in transactions". coz they >> were running very

Re: [External] Re: FDW, too long to run explain

2019-02-17 Thread Vijaykumar Jain
Hey Jeff, yes, we now relaxed the idle in transaction setting to 15 mins. i was hesitant to increase the settings as it blocked auto vaccum. We use hot_standby_feedback = true also as we split reads/writes and allow long running queries on read replicas, this too affects auto vaccum. so overall,

Re: [External] Re: FDW, too long to run explain

2019-02-17 Thread Jeff Janes
On Sun, Feb 17, 2019 at 1:52 PM Vijaykumar Jain wrote: > Assuming your questions as 1,2,3, please find my answers below. > > 1)"explain" on foreign servers run as "idle in transactions". coz they > were running very long (in the order of some minutes) , pgbouncer (in tx > level pooling) setting

Re: FDW, too long to run explain

2019-02-17 Thread Tom Lane
Jeff Janes writes: > A question for the PostgreSQL hackers would be, Is it necessary and > desirable that the EXPLAIN be issued in the same transaction as the > eventual DECLARE and FETCHes? I don't think it is. It seems like a good idea to me. I certainly don't think "I've got an

Re: FDW, too long to run explain

2019-02-17 Thread Jeff Janes
On Mon, Feb 4, 2019 at 2:15 AM Vijaykumar Jain wrote: > > now we have some long running queries via FDW that take minutes and get > killed explain runs as idle in transaction on remote servers. > Are you saying the EXPLAIN itself gets killed, or execution of the plan generated based on the

Re: [External] Re: FDW, too long to run explain

2019-02-17 Thread Vijaykumar Jain
I am yet to figure out the reason, what we have done is implement fake columns to represent samples and giving them random numbers and keeping other bulls to fake limit. Most of the queries that were impacted were the ones that did not push order by and limit to foreign servers. I am also trying

Re: FDW, too long to run explain

2019-02-17 Thread auxsvr
Related to this question: Postgresql cursors are in most cases I've tried extremely slow. The cause is as described in my previous answer, in my experience. Is there any plan or way to improve this situation? For example, for FDW one would expect the plan on the remote side to be similar, if

FDW, too long to run explain

2019-02-03 Thread Vijaykumar Jain
Hi, with pg v10.1 I have a setup enabled as below. 7 shards ( 1RW, 2 RO ) they all are fronted by FDW talking to each other. we use writes directly to shards, and reads via FDW from all shards (RO) our DB size is ~ 500GB each shard, and tables are huge too. 1 table ~ 200GB, 1 ~55GB, 1 ~40GB