[PERFORM] Slow query help

2016-01-06 Thread Almir de Oliveira Duarte Junior
Hi, I ask your help to solve a slow query which is taking more than 14 seconds to be executed. Maybe I am asking too much both from you and specially from postgresql, as it is really huge, envolving 16 tables. Explain: http://explain.depesz.com/s/XII9 Schema: http://adj.com.br/erp/data_schem

Re: [PERFORM] Queries intermittently slow

2016-01-06 Thread Jim Nasby
On 1/6/16 12:01 PM, Scott Rankin wrote: I guess we’re back to lock contention? Is there by chance an anti-wraparound vacuum happening on that table? Actually, for that matter... if autovacuum is hitting that table it's locking could be causing problems, and it won't release it's locks until

Re: [PERFORM] Queries intermittently slow

2016-01-06 Thread Scott Rankin
> >So what seems more likely after more thought is that the pages are staying >in our shared buffer arena just fine, but the kernel is randomly choosing >to swap out parts of the arena, and the delays correspond to swap-in >waits. (There would still have to be a mighty crummy disk subsystem >under

Re: [PERFORM] Materialized view performance problems

2016-01-06 Thread Andreas Kretschmer
Tom McLoughlin wrote: > Thank you very much for your help. > > It's difficult for me to run analyse explain for the query given because it > takes so long. However, the query below has a similar structure but has less > data to process. Seems okay, but it's better to analyse that for the real q

Re: [PERFORM] Queries intermittently slow

2016-01-06 Thread Tom Lane
I wrote: > A possible theory is that the slow cases represent times when the desired > page is not in cache, but you'd have to have a seriously overloaded disk > subsystem for a disk fetch to take hundreds of ms. Unless maybe this is > running on some cloud service with totally unspecified I/O ban

Re: [PERFORM] Queries intermittently slow

2016-01-06 Thread Scott Rankin
> >Hm, well, given that you are able to capture instances of the behavior >in EXPLAIN ANALYZE, I'd suggest trying EXPLAIN (ANALYZE,BUFFERS). >That will tell you the number of pages it found in shared buffers vs. >having to read them. Now, a "read" just means we had to ask the kernel, >not necessar

Re: [PERFORM] Queries intermittently slow

2016-01-06 Thread Tom Lane
Scott Rankin writes: > On 1/6/16, 10:38 AM, "Tom Lane" wrote: >> A possible theory is that the slow cases represent times when the desired >> page is not in cache, but you'd have to have a seriously overloaded disk >> subsystem for a disk fetch to take hundreds of ms. Unless maybe this is >> run

Re: [PERFORM] Queries intermittently slow

2016-01-06 Thread Scott Rankin
On 1/6/16, 10:38 AM, "Tom Lane" wrote: > >A possible theory is that the slow cases represent times when the desired >page is not in cache, but you'd have to have a seriously overloaded disk >subsystem for a disk fetch to take hundreds of ms. Unless maybe this is >running on some cloud service wi

Re: [PERFORM] Queries intermittently slow

2016-01-06 Thread Tom Lane
Scott Rankin writes: > Thanks for the update. The query in question is a pretty simple one - it > joins 3 tables, all of which are static - they don’t have any writes being > done against them. They have very few rows, and the query plan for them > indicates that they are all sequential scan

Re: [PERFORM] Queries intermittently slow

2016-01-06 Thread Scott Rankin
On 1/6/16, 10:19 AM, "Tom Lane" wrote: >Scott Rankin writes: >> I’m trying to track down why some queries on my database system are >> intermittently much slower than usual. I have some queries that run, on >> average, 2-3ms, and they run at a rate of about 10-20 queries/second. >> However

Re: [PERFORM] Queries intermittently slow

2016-01-06 Thread Tom Lane
Scott Rankin writes: > I’m trying to track down why some queries on my database system are > intermittently much slower than usual. I have some queries that run, on > average, 2-3ms, and they run at a rate of about 10-20 queries/second. > However, every 3-5 seconds, one of the queries will

[PERFORM] Queries intermittently slow

2016-01-06 Thread Scott Rankin
Hi all, I’m trying to track down why some queries on my database system are intermittently much slower than usual. I have some queries that run, on average, 2-3ms, and they run at a rate of about 10-20 queries/second. However, every 3-5 seconds, one of the queries will be 500-100ms. This is

Re: [PERFORM] Materialized view performance problems

2016-01-06 Thread Tom McLoughlin
Thank you very much for your help. It's difficult for me to run analyse explain for the query given because it takes so long. However, the query below has a similar structure but has less data to process. create materialized view temp_camp_perf_unaggr as select account_websites.id as website_id

Re: [PERFORM] Materialized view performance problems

2016-01-06 Thread Andreas Kretschmer
> Tom McLoughlin hat am 6. Januar 2016 um 09:08 > geschrieben: > > > > As you can see below it's a big query, and I didn't want to overwhelm > everyone with the schema, so let me know what bits you might need to help! > > Any help improving the performance will be greatly appreciated. can

[PERFORM] Materialized view performance problems

2016-01-06 Thread Tom McLoughlin
Hello, I'm having trouble with the performance from a query used to create a materialized view. I need to be able to build the keyword_performance_flat_matview view in around 2-4 hours overnight. However, it currently takes in excess of 24 hours. I'm wondering if there is anything I can do to imp