[PERFORM] probably cause (and fix) for floating-point assist faults on itanium

2011-11-17 Thread Greg Matthews
Hi folks, I'm running PG 8.3.15 on an itanium box and was seeing lots of floating-point assist faults by the kernel. Searched around, found a couple references/discussions here and there: http://archives.postgresql.org/pgsql-general/2008-08/msg00244.php http://archives.postgresql.org/pgsql-pe

Re: [PERFORM] SSD options, small database, ZFS

2011-11-17 Thread Arjen van der Meijden
On 18-11-2011 4:44 CSS wrote: Resurrecting this long-dormant thread... Btw, the 5500 and 5600 Xeons are normally more efficient with a multiple of 6 ram-modules, so you may want to have a look at 24GB (6x4), 36GB (6x4+6x2) or 48GB (12x4 or 6x8) RAM. Thanks - I really had a hard time wrappi

Re: [PERFORM] SSD options, small database, ZFS

2011-11-17 Thread CSS
Resurrecting this long-dormant thread... On Oct 14, 2011, at 6:41 AM, Arjen van der Meijden wrote: > On 14-10-2011 10:23, CSS wrote: >> -I'm calling our combined databases at 133GB "small", fair >> assumption? -Is there any chance that a server with dual quad core >> xeons, 32GB RAM, and 2 or 4

Re: [PERFORM] index usage for min() vs. "order by asc limit 1"

2011-11-17 Thread Ben Chobot
On Nov 17, 2011, at 5:20 PM, Steve Atkins wrote: > > I don't think you want the group by in that first query. Heh, I tried to simply the example, but in reality that = becomes an in clause of multiple values. So the group by is needed. >> >> >> postgres=# explain analyze select min(id) from

Re: [PERFORM] index usage for min() vs. "order by asc limit 1"

2011-11-17 Thread Steve Atkins
On Nov 17, 2011, at 5:12 PM, Ben Chobot wrote: > I have two queries in PG 9.1. One uses an index like I would like, the other > does not. Is this expected behavior? If so, is there any way around it? I don't think you want the group by in that first query. Cheers, Steve > > > postgres=#

[PERFORM] index usage for min() vs. "order by asc limit 1"

2011-11-17 Thread Ben Chobot
I have two queries in PG 9.1. One uses an index like I would like, the other does not. Is this expected behavior? If so, is there any way around it? postgres=# explain analyze select min(id) from delayed_jobs where strand='sis_batch:account:15' group by strand;

Re: [PERFORM] external sort performance

2011-11-17 Thread Josh Berkus
> The first log statement is expected. The second log statement, however, isn't. > The total table size is (as noted earlier) about 5GB and, in fact, fit > into one nice hash table (approx 15GB in size). > Is the sorting that is necessary for index creation unable to use a > hash table? (This is a

Re: [PERFORM] external sort performance

2011-11-17 Thread Tom Lane
Jon Nelson writes: > Is the sorting that is necessary for index creation unable to use a > hash table? (This is a standard btree index). Hash aggregation isn't sorting --- it's only useful for grouping. regards, tom lane -- Sent via pgsql-performance mailing list (pgsql

Re: [PERFORM] external sort performance

2011-11-17 Thread Jon Nelson
A follow-up question. Even with both work_mem and maintenance_work_mem equal to 16GB, I see this: LOG: 0: begin index sort: unique = f, workMem = 16777216, randomAccess = f and shortly thereafter: LOG: 0: switching to external sort with 59919 tapes: CPU 2.59s/13.20u sec elapsed 16.85 sec

Re: [PERFORM] external sort performance

2011-11-17 Thread Jon Nelson
I'll try to compile multiple questions/answers into a single response. On Thu, Nov 17, 2011 at 11:16 AM, Claudio Freire wrote: > On Thu, Nov 17, 2011 at 2:10 PM, Jon Nelson wrote: >> What sorts of things should I be looking at to improve the performance >> of this query? Is my interpretation of

Re: [PERFORM] external sort performance

2011-11-17 Thread Tom Lane
Jon Nelson writes: > This is on PG 8.4.8 on Linux, 16GB of "real" RAM. > Most recently, I enabled trace_sort, disabled hash aggregation[1], and > set a large work_mem (normally very small, in this case I tried > anything from 8MB to 256MB. I even tried 1GB and 2GB). FWIW, I think hash aggregation

Re: [PERFORM] external sort performance

2011-11-17 Thread Craig James
On 11/17/11 9:10 AM, Jon Nelson wrote: I have one query which does not run very often. Sometimes it may be months between runs. However, when it does get executed, it scans approximately 100 identically-structured tables (a form of partitioning), extracts and groups on a subset of the columns, an

Re: [PERFORM] external sort performance

2011-11-17 Thread Claudio Freire
On Thu, Nov 17, 2011 at 2:10 PM, Jon Nelson wrote: > What sorts of things should I be looking at to improve the performance > of this query? Is my interpretation of that log line totally off base? You'll have to post some more details. Like a query and an explain/explain analyze. Memory consumpt

[PERFORM] external sort performance

2011-11-17 Thread Jon Nelson
I have one query which does not run very often. Sometimes it may be months between runs. However, when it does get executed, it scans approximately 100 identically-structured tables (a form of partitioning), extracts and groups on a subset of the columns, and creates a new table. The individual tab

Re: [PERFORM] Performance question 83 GB Table 150 million rows, distinct select

2011-11-17 Thread Claudio Freire
On Thu, Nov 17, 2011 at 11:17 AM, Aidan Van Dyk wrote: > But remember, you're doing all that in a single query.  So your disk > subsystem might even be able to perform even more *througput* if it > was given many more concurrent request.  A big raid10 is really good > at handling multiple concurre

Re: [PERFORM] Performance question 83 GB Table 150 million rows, distinct select

2011-11-17 Thread Tomas Vondra
On 17 Listopad 2011, 15:17, Aidan Van Dyk wrote: > With a single query, the query can only run as fast as the single > stream of requests can be satisfied. And as the next read is issued > as soon as the previous is done (the kernel readahead/buffering the > seq scan helps here), your iostat is go

Re: [PERFORM] Performance question 83 GB Table 150 million rows, distinct select

2011-11-17 Thread Aidan Van Dyk
On Thu, Nov 17, 2011 at 12:23 AM, Tory M Blue wrote: >> What do you mean by "nothing"? There are 3060 reads/s, servicing each one >> takes 0.33 ms - that means the drive is 100% utilized. >> >> The problem with the iostat results you've posted earlier is that they >> either use "-xd" or none of t