Re: [PERFORM] Postgres 8.3, four times slower queries?
On Thu, Mar 5, 2009 at 12:58 PM, Joshua D. Drake j...@commandprompt.comwrote: What happens if you do this: SET cpu_tuple_cost TO '0.5'; SET cpu_index_tuple_cost TO '0.5'; EXPLAIN ANALYZE 8.3 query Next try this: SET cpu_tuple_cost TO '0.5'; SET cpu_index_tuple_cost TO '0.5'; SET seq_page_cost TO '4.0'; SET random_page_cost TO '1.0'; EXPLAIN ANALYZE 8.3 query And then this: SET cpu_tuple_cost TO '0.5'; SET cpu_index_tuple_cost TO '0.5'; SET seq_page_cost TO '4.0'; SET random_page_cost TO '1.0'; SET effective_cache_size TO '3000MB'; EXPLAIN ANALYZE 8.3 query These three are pretty much the same in terms of performance. I stayed with the first one (cpu_tuple_cost = 0.5 and cpu_index_tuple_cost = 0.5). As shown earlier, it gives a result similar or slightly better than 8.2.12 in terms of performance and response time. The explain analyze shows that the query no longer causes postgreSQL to uses hashes, but indexes instead which boosted the performance of the query from ~1200 ms to ~600 ms. Thank you everyone for all the help and feedback on this issue.
Re: [PERFORM] Postgres 8.3, four times slower queries?
Matching query plans with numerics changed to integers. I sent the wrong query plans earlier 8.3.3: 1195 ms 8.2.12: 611 ms QUERY PLAN - Unique (cost=6986.01..6986.02 rows=1 width=20) (actual time=611.061..611.061 rows=0 loops=1) - Sort (cost=6986.01..6986.02 rows=1 width=20) (actual time=611.053..611.053 rows=0 loops=1) Sort Key: t8.id - Nested Loop (cost=955.90..6986.00 rows=1 width=20) (actual time=610.942..610.942 rows=0 loops=1) Join Filter: ((t2.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t3.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t6.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t7.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t8.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t9.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t10.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t11.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text))) - Nested Loop (cost=955.90..4989.27 rows=523 width=76) (actual time=28.274..548.664 rows=1104 loops=1) - Hash Join (cost=955.90..3195.56 rows=526 width=52) (actual time=28.123..520.620 rows=1104 loops=1) Hash Cond: (t6.invoice_id = t7.id) - Seq Scan on payment t6 (cost=0.00..2002.69 rows=61789 width=16) (actual time=0.026..253.360 rows=60920 loops=1) Filter: (active 0) - Hash (cost=954.15..954.15 rows=140 width=44) (actual time=25.795..25.795 rows=332 loops=1) - Nested Loop (cost=0.00..954.15 rows=140 width=44) (actual time=0.546..24.364 rows=332 loops=1) - Nested Loop (cost=0.00..562.62 rows=100 width=40) (actual time=0.471..15.759 rows=336 loops=1) - Nested Loop (cost=0.00..74.40 rows=156 width=28) (actual time=0.233..6.404 rows=376 loops=1) - Nested Loop (cost=0.00..45.04 rows=4 width=20) (actual time=0.157..0.324 rows=4 loops=1) - Index Scan using idx_department_du on department t10 (cost=0.00..7.30 rows=6 width=12) (actual time=0.076..0.108 rows=7 loops=1) Index Cond: (company_id = 250893) Filter: (active 0) - Index Scan using idx_project_department_id on project t2 (cost=0.00..6.28 rows=1 width=16) (actual time=0.016..0.018 rows=1 loops=7) Index Cond: (t2.department_id = t10.id) Filter: (active 0) - Index Scan
Re: [PERFORM] Postgres 8.3, four times slower queries?
On Thu, Mar 5, 2009 at 12:58 PM, Joshua D. Drake j...@commandprompt.comwrote: What happens if you do this: SET cpu_tuple_cost TO '0.5'; SET cpu_index_tuple_cost TO '0.5'; EXPLAIN ANALYZE 8.3 query Right now, I'm getting very good results with the above. I'm still running additional tests but I'll keep you guys updated. I've attached the new explain analyze. SET SET QUERY PLAN - Unique (cost=17397.17..17397.18 rows=2 width=20) (actual time=621.865..621.865 rows=0 loops=1) - Sort (cost=17397.17..17397.18 rows=2 width=20) (actual time=621.857..621.857 rows=0 loops=1) Sort Key: t8.id Sort Method: quicksort Memory: 17kB - Nested Loop (cost=0.00..17397.16 rows=2 width=20) (actual time=621.825..621.825 rows=0 loops=1) Join Filter: ((t2.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t3.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t6.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t7.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t8.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t9.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t10.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t11.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text))) - Nested Loop (cost=0.00..14615.82 rows=537 width=76) (actual time=51.092..556.405 rows=1104 loops=1) - Nested Loop (cost=0.00..12007.84 rows=541 width=52) (actual time=46.530..523.869 rows=1104 loops=1) - Nested Loop (cost=0.00..2229.06 rows=147 width=44) (actual time=42.642..501.168 rows=332 loops=1) - Nested Loop (cost=0.00..1530.56 rows=114 width=40) (actual time=39.485..489.224 rows=336 loops=1) - Nested Loop (cost=0.00..757.31 rows=177 width=28) (actual time=10.877..473.774 rows=376 loops=1) - Nested Loop (cost=0.00..69.07 rows=4 width=20) (actual time=6.463..6.640 rows=4 loops=1) - Index Scan using idx_department_du on department t10 (cost=0.00..14.72 rows=7 width=12) (actual time=6.141..6.173 rows=7 loops=1) Index Cond: (company_id = 250893) Filter: (active 0) - Index Scan using idx_project_department_id on project t2 (cost=0.00..7.26 rows=1 width=16) (actual time=0.051..0.053 rows=1 loops=7) Index Cond: (t2.department_id = t10.id) Filter: (t2.active 0) - Index Scan using idx_project_type_project_id on project_type t11 (cost=0.00..133.37 rows=77 width=16) (actual time=1.112..116.094 rows=94
Re: [PERFORM] Postgres 8.3, four times slower queries?
On Tue, Mar 3, 2009 at 5:34 PM, Tom Lane t...@sss.pgh.pa.us wrote: Comparisons on numerics aren't terribly fast though (in either release). I wonder whether you could change the key columns to int or bigint. I changed the affected columns from numeric to integers and I was unable to get any performance gain: 8.3.3: 1195 ms 8.2.12: 611 ms I've attached the new query plans. Are you doing something to force the join order, like running with a small join_collapse_limit setting? If so maybe you shouldn't. No, we left the join_collapse_limit to the default 8. We tried a higher value, but there was no difference in performance. I'll post the query and the table descriptions in separate messages to the list to avoid my mail from being rejected for exceeding the size limit :) QUERY PLAN - Unique (cost=6986.01..6986.02 rows=1 width=20) (actual time=611.061..611.061 rows=0 loops=1) - Sort (cost=6986.01..6986.02 rows=1 width=20) (actual time=611.053..611.053 rows=0 loops=1) Sort Key: t8.id - Nested Loop (cost=955.90..6986.00 rows=1 width=20) (actual time=610.942..610.942 rows=0 loops=1) Join Filter: ((t2.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t3.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t6.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t7.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t8.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t9.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t10.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t11.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text))) - Nested Loop (cost=955.90..4989.27 rows=523 width=76) (actual time=28.274..548.664 rows=1104 loops=1) - Hash Join (cost=955.90..3195.56 rows=526 width=52) (actual time=28.123..520.620 rows=1104 loops=1) Hash Cond: (t6.schedule_id = t7.id) - Seq Scan on bundle t6 (cost=0.00..2002.69 rows=61789 width=16) (actual time=0.026..253.360 rows=60920 loops=1) Filter: (active 0) - Hash (cost=954.15..954.15 rows=140 width=44) (actual time=25.795..25.795 rows=332 loops=1) - Nested Loop (cost=0.00..954.15 rows=140 width=44) (actual time=0.546..24.364 rows=332 loops=1) - Nested Loop (cost=0.00..562.62 rows=100 width=40) (actual time=0.471..15.759 rows=336 loops=1) - Nested Loop (cost=0.00..74.40 rows=156 width=28) (actual time=0.233..6.404 rows=376 loops=1) - Nested Loop (cost=0.00..45.04 rows=4 width=20) (actual time=0.157..0.324 rows=4 loops=1) - Index Scan using idx_day_part_du on day_part t10 (cost=0.00..7.30 rows=6 width=12) (actual
Re: [PERFORM] Postgres 8.3, four times slower queries?
Query and first part of the table descriptions Query: explain analyze select distinct on (t8.id) t8.id, t8.payment_id, t8.amount_id, t8.active, t8.organization_id from project t2, project_invoice t3, payment t6, invoice t7, payment_amount t8, amount t9, department t10, project_type t11 where t10.company_id = 250893 and t2.department_id = t10.id and t3.id = t7.owner_resource_id and t2.id = t11.project_id and t11.slot_id = t3.id and t6.invoice_id = t7.id and t6.id = t8.payment_id and t8.amount_id = t9.id and t2.active 0 and t3.active 0 and t6.active 0 and t7.active 0 and t10.active 0 and t11.active 0 and (t2.not_modified_since = to_timestamp(E'2009-02-24T11:19:57', E'/MM/DD HH:MI:SS') or t3.not_modified_since = to_timestamp(E'2009-02-24T11:19:57', E'/MM/DD HH:MI:SS') or t6.not_modified_since = to_timestamp(E'2009-02-24T11:19:57', E'/MM/DD HH:MI:SS') or t7.not_modified_since = to_timestamp(E'2009-02-24T11:19:57', E'/MM/DD HH:MI:SS') or t8.not_modified_since = to_timestamp(E'2009-02-24T11:19:57', E'/MM/DD HH:MI:SS') or t9.not_modified_since = to_timestamp(E'2009-02-24T11:19:57', E'/MM/DD HH:MI:SS') or t10.not_modified_since = to_timestamp(E'2009-02-24T11:19:57', E'/MM/DD HH:MI:SS') or t11.not_modified_since = to_timestamp(E'2009-02-24T11:19:57', E'/MM/DD HH:MI:SS')); Table public.payment Column |Type | Modifiers ---+-+ id| integer | not null name | character varying(255) | active| integer | not null default 1 status| numeric | not null default 0 last_status_change| date| not_modified_since| timestamp without time zone | organization_id | integer | added_user_id | numeric | added_tm | timestamp without time zone | edited_user_id| numeric | edited_tm | timestamp without time zone | folder_id | integer | max_duration_msec | numeric | tax_id| integer | customer_id | integer | allow_custom_duration | numeric | loop_tax_id | integer | loop_weight | numeric | fullbill | numeric | invoice_id| integer | trigger_tax_id| integer | Indexes: payment_pkey PRIMARY KEY, btree (id) idx_payment_organization_id btree (organization_id) idx_payment_lower_name btree (lower(name::text)) idx_payment_nms btree (not_modified_since) idx_payment_invoice_id btree (invoice_id) Foreign-key constraints: payment_tax_id_fkey FOREIGN KEY (tax_id) REFERENCES tax(id) payment_folder_id_fkey FOREIGN KEY (folder_id) REFERENCES folder(id) payment_customer_id_fkey FOREIGN KEY (customer_id) REFERENCES customer(id) payment_loop_tax_id_fkey FOREIGN KEY (loop_tax_id) REFERENCES tax(id) payment_invoice_id_fkey FOREIGN KEY (invoice_id) REFERENCES invoice(id) payment_trigger_tax_id_fkey FOREIGN KEY (trigger_tax_id) REFERENCES tax(id) Inherits: resource Table public.payment_amount Column |Type | Modifiers +-+ id | integer | not null name | character varying(255) | active | integer | not null default 1 status | numeric | not null default 0 last_status_change | date| not_modified_since | timestamp without time zone |
Re: [PERFORM] Postgres 8.3, four times slower queries?
2nd part of table descriptions Table public.company Column|Type | Modifiers --+-+ id | integer | not null name | character varying(255) | active | integer | not null default 1 status | numeric | not null default 0 last_status_change | date| not_modified_since | timestamp without time zone | organization_id | integer | added_user_id| numeric | added_tm | timestamp without time zone | edited_user_id | numeric | edited_tm| timestamp without time zone | folder_id| integer | company_type_id | integer | not null timezone | character varying(255) | host_bill_count | numeric | enforce_departments | boolean | not null default false zipcode | character varying(255) | address | character varying(255) | Indexes: company_pkey PRIMARY KEY, btree (id) idx_company_organization_id btree (organization_id) idx_company_lower_name btree (lower(name::text)) idx_company_nms btree (not_modified_since) Foreign-key constraints: company_folder_id_fkey FOREIGN KEY (folder_id) REFERENCES folder(id) company_company_type_id_fkey FOREIGN KEY (company_type_id) REFERENCES company_type(id) Inherits: resource Table public.project_invoice Column |Type | Modifiers +-+ id | integer | not null name | character varying(255) | active | integer | not null default 1 status | numeric | not null default 0 last_status_change | date| not_modified_since | timestamp without time zone | organization_id| integer | added_user_id | numeric | added_tm | timestamp without time zone | edited_user_id | numeric | edited_tm | timestamp without time zone | folder_id | integer | start_date | date| end_date | date| owner_resource_id | numeric | duration | numeric | saturation | numeric | Indexes: project_invoice_pkey PRIMARY KEY, btree (id) idx_project_invoice_organization_id btree (organization_id) idx_project_invoice_lower_name btree (lower(name::text)) idx_project_invoice_nms btree (not_modified_since) idx_project_invoice_owner_resource_id btree (owner_resource_id) Inherits: resource Table public.resource Column |Type | Modifiers +-+ id | integer | not null name | character varying(255) | active | integer | not null default 1 status | numeric | not null default 0 last_status_change | date| not_modified_since | timestamp without time zone | organization_id| integer | added_user_id | numeric | added_tm | timestamp without time zone | edited_user_id | numeric | edited_tm | timestamp without time zone | folder_id | integer | Indexes: resource_pkey PRIMARY KEY, btree (id) idx_organization_id btree (organization_id) idx_resource_lower_name btree (lower(name::text)) Foreign-key constraints: resource_folder_id_fkey FOREIGN KEY (folder_id) REFERENCES folder(id) Table public.invoice Column |Type | Modifiers +-+ id | integer | not null name | character varying(255) | active | integer | not null default 1 status | numeric | not null default 0 last_status_change | date| not_modified_since | timestamp without time zone | organization_id| integer | added_user_id | numeric | added_tm | timestamp
Re: [PERFORM] Postgres 8.3, four times slower queries?
On Mon, Mar 2, 2009 at 10:23 PM, Tom Lane t...@sss.pgh.pa.us wrote: Are you sure you are comparing apples to apples here? Same configure options for the builds, same parameter values in postgresql.conf, both databases ANALYZEd, etc? And are they running on the same hardware? Thank you for looking at this Tom. Yes, we have made sure we are comparing apples to apples here. The postgresql.confs are identical, as are the configure flags: --disable-debug --enable-shared --enable-thread-safety --with-perl --with-pam --without-docdir --without-tcl --without-python --without-krb5 --without-ldap --without-bonjour --enable-integer-datetimes --prefix=/opt/postgresql However, the db was not analyzed. I'll attached the new explain analyze of the queries with the db analyzed, but 8.2 still beats 8.3. The tests are both being run on the same machine, a Quad-core AMD Opteron Processor 2212 (each with 1024 KB cache) and 4GB of RAM. I find it telling that the query plan differs so much between postgres 8.2. and 8.3. For example, why does the 8.3. planner choose to perform so many seq scans? I know seq scans are faster than index scans for small tables, but these tables have 60K+ rows... surely an index scan would have been a better choice here? If you look at the 8.2. query plan, it is very clean in comparison, index scans all the way through. I can't help but think the 8.3 planner is simply failing to make the right choices in our case. Another question would be, why are there so many hash joins in the 8.3 plan now? All our indexes are btrees... Any light that can be shed on what going on with the 8.3. planner would be much appreciated. Thanks in advance.
Re: [PERFORM] Postgres 8.3, four times slower queries?
On Tue, Mar 3, 2009 at 12:38 PM, da...@lang.hm wrote: if you haven't done a vaccum analyse on either installation then postgres' idea of what sort of data is in the database is unpredictable, and as a result it's not surprising that the two systems guess differently about what sort of plan is going to be most efficiant. try doing vaccum analyse on both databases and see what the results are. David Lang These are the results with vacuum analyze: 8.2.12: 624.366 ms 8.3.3: 1273.601 ms QUERY PLAN - Unique (cost=9832.91..9832.92 rows=2 width=52) (actual time=623.808..623.808 rows=0 loops=1) - Sort (cost=9832.91..9832.92 rows=2 width=52) (actual time=623.799..623.799 rows=0 loops=1) Sort Key: t8.id - Nested Loop (cost=0.00..9832.90 rows=2 width=52) (actual time=623.467..623.467 rows=0 loops=1) Join Filter: ((t2.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t3.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t6.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t7.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t8.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t9.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t10.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t11.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text))) - Nested Loop (cost=0.00..7381.49 rows=598 width=108) (actual time=75.634..556.642 rows=1104 loops=1) - Nested Loop (cost=0.00..5087.76 rows=602 width=59) (actual time=71.744..523.690 rows=1104 loops=1) - Nested Loop (cost=0.00..1171.66 rows=165 width=51) (actual time=66.427..499.798 rows=332 loops=1) - Nested Loop (cost=0.00..684.77 rows=117 width=54) (actual time=38.266..440.024 rows=336 loops=1) - Nested Loop (cost=0.00..84.08 rows=182 width=35) (actual time=16.785..402.046 rows=376 loops=1) - Nested Loop (cost=0.00..52.33 rows=4 width=27) (actual time=11.355..11.547 rows=4 loops=1) - Index Scan using idx_day_part_du on day_part t10 (cost=0.00..8.28 rows=7 width=19) (actual time=0.713..0.748 rows=7 loops=1) Index Cond: (display_unit_id = 250893::numeric) Filter: (active 0::numeric) - Index Scan using idx_skin_day_part_id on skin t2 (cost=0.00..6.28 rows=1 width=30) (actual time=1.526..1.529 rows=1 loops=7) Index Cond: (t2.day_part_id = t10.id) Filter: (active 0::numeric) - Index Scan using idx_skin_slot_skin_id on skin_slot t11
[PERFORM] Postgres 8.3, four times slower queries?
Hi, We are currently running postgres 8.2 and are evaluating the upgrade to 8.3. Some of our tests are indicating that postgresql 8.3 is actually degrading the performance of some of our queries by a factor of 10 or more. The queries in question are selects that are heavy on joins (~10 tables) with a lot of timestamp-based conditions in where clauses. The tables and queries are tuned, that is, there is no issue with the table structure, or missing indexes. This is a side-by-side query performance measurement between 8.2 and 8.3 with an identical dataset and schema. 8.2.12 8.3.3 Time (ms)Time (ms) 1st 2nd1st 2nd time time time time Query 1 759 1303294 1758 attached you will find the explain analyze for this query. Any insight into this issue would be very appreciated. Thanks. QUERY PLAN - Unique (cost=8251.47..8251.48 rows=1 width=37) (actual time=3294.104..3294.104 rows=0 loops=1) - Sort (cost=8251.47..8251.48 rows=1 width=37) (actual time=3294.096..3294.096 rows=0 loops=1) Sort Key: t8.id Sort Method: quicksort Memory: 17kB - Hash Join (cost=5509.94..8251.46 rows=1 width=37) (actual time=3294.030..3294.030 rows=0 loops=1) Hash Cond: (t9.id = t8.content_id) Join Filter: ((t2.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t3.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t6.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t7.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t8.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t9.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t10.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text)) OR (t11.not_modified_since = to_timestamp('2009-02-24T11:19:57'::text, '/MM/DD HH:MI:SS'::text))) - Seq Scan on content t9 (cost=0.00..2272.26 rows=68026 width=16) (actual time=3.891..377.945 rows=68045 loops=1) - Hash (cost=5498.91..5498.91 rows=882 width=93) (actual time=2625.501..2625.501 rows=1104 loops=1) - Hash Join (cost=3590.26..5498.91 rows=882 width=93) (actual time=2241.886..2620.141 rows=1104 loops=1) Hash Cond: (t8.bundle_id = t6.id) - Seq Scan on bundle_content t8 (cost=0.00..1638.89 rows=69589 width=45) (actual time=5.160..334.263 rows=69606 loops=1) - Hash (cost=3579.17..3579.17 rows=887 width=56) (actual time=1998.681..1998.681 rows=1104 loops=1) - Hash Join (cost=1317.36..3579.17 rows=887 width=56) (actual time=1367.063..1993.592 rows=1104 loops=1) Hash Cond: (t6.schedule_id = t7.id) - Seq Scan on bundle t6 (cost=0.00..2023.34 rows=61227 width=24) (actual