Thanks for the suggestion, however; this query runs in under 2 minutes (big data set) on the server it originated from, and I’m trying to ascertain why there is such a difference in performance between what should be two like for like server, as there are many other queries of a similar nature that have bigger sort operations to perform, and I honestly don’t want to have to re-write them all.
Setting enable_hashagg = off causes the query to complete, however; it means the next batch queries never completes (left it > 8 hours) when ordinarily it’s done in 5 minutes. Thanks… Martin. From: [email protected] [mailto:[email protected]] Sent: 07 April 2011 11:20 To: French, Martin; [email protected] Subject: Re: [ADMIN] Out Of Memory 8.1 Have you tried moving the FROM ... WHERE ... into a sub-select? GROUP BY uses HAVING, not WHERE. ----- Reply message ----- From: "French, Martin" <[email protected]> Date: Wed, Apr 6, 2011 7:44 pm Subject: [ADMIN] Out Of Memory 8.1 To: <[email protected]> Hi All, I am having problems with a query on 8.1 running on RHEL 5.4 16GB RAM Linux pgsql3 2.6.18-164.el5PAE #1 SMP Tue Aug 18 15:59:11 EDT 2009 i686 i686 i386 GNU/Linux 2 x Xeon X5650 (2.67GHz 6 Cores) Disks are on PERC 6 controller in RAID 10 Postgresql.conf: # - Memory - shared_buffers = 327680 work_mem = 98394 maintenance_work_mem = 983040 The query: 2011-04-06 23:23:03 BST <app.charisma><26083>LOG: statement: CREATE TABLE medusa.cmi_group_free AS SELECT stkl_stockno, (SUM(stkphys) - (SUM(stkalloc) + SUM(stkbacko))) as group_free FROM charisma.sk_stklfl WHERE stkl_stockno NOT LIKE 'ZZ%' AND stkl_stockno ~ '^([A-z&]{2,3})([0-9][0-9][0-9])([0-9][0-9][0-9][0-9][A-z0-9]{1,3})$' GROUP BY stkl_stockno; The explain (cannot explain analyze, or Postgres runs out of memory again) 'HashAggregate (cost=2731947.55..2731947.57 rows=1 width=38)' ' -> Seq Scan on stkl_rec (cost=0.00..2731947.54 rows=1 width=38)' ' Filter: (((stkl_comp)::text = 'A'::text) AND ((stkl_stockno)::text ~ '^(TK[A-Za-z0-9][0-9]{3}(?:[0-9]{5}(?:[0-9]{3,4})?)?|NSP[0-9]{3}([0-9]{4 })?|Z[MZ][0-9]{8,9}|LSP[0-9]{7}[A-Za-z0-9]|[A-Z][A-Z&][A-Z][0-9]{7}[A-Z0 -9])$'::text) AND ((stkl_stockno)::text !~~ 'ZZ%'::text) AND ((stkl_stockno)::text ~ '^([A-z&]{2,3})([0-9][0-9][0-9])([0-9][0-9][0-9][0-9][A-z0-9]{1,3})$'::t ext))' The table has approximately 6.9 million rows It's not the OOM that's doing this as vm.overcommit_memory = 2 and vm.overcommit_ratio = 90 nor is postgres swapping to disk. Can anyone help me here? The same query on exactly the same data on an older machine running the same setup except that it's RHEL 5.0 runs fine! Thanks. Martin ___________________________________________________ This email is intended for the named recipient. The information contained in it is confidential. You should not copy it for any purposes, nor ___________________________________________________ This email is intended for the named recipient. The information contained in it is confidential. You should not copy it for any purposes, nor disclose its contents to any other party. If you received this email in error, please notify the sender immediately via email, and delete it from your computer. Any views or opinions presented are solely those of the author and do not necessarily represent those of the company. PCI Compliancy: Please note, we do not send or wish to receive banking, credit or debit card information by email or any other form of communication. Cromwell Tools Limited, PO Box 14, 65 Chartwell Drive Wigston, Leicester LE18 1AT. Tel 0116 2888000 Registered in England and Wales, Reg No 00986161 VAT GB 115 5713 87 900 __________________________________________________
