Re: [PERFORM] (Re)-indexing on updates

2005-08-22 Thread Yves Vindevogel
The option with T1: A B C and T2 A D (to avoid the updates) works very well with a simple query Insert into T2 (A, D) select A, functionToGetD from T1 left join T2 on T1.A = T2.A where T2.A is null The above gives me the new records for those where D was not filled yet. Since they are all

Re: [PERFORM] sustained update load of 1-2k/sec

2005-08-22 Thread Mark Cotner
Thanks again everyone for the excellent suggestions. I looked into IO::Reactor, but after a few hours of fiddling decided I was getting the kind of performance I wanted from using a slightly more than modest number of threads and decided(due to dev timelines) to come back to patching the SNMP

[PERFORM] MemoryContextSwitchTo during table scan?

2005-08-22 Thread Jignesh Shah
Hello, I am running PostgreSQL 8.0.x on Solaris 10 AMD64. My Tablesize for this test is about 80G. When I run a query on a column which is not indexed, I get a full table scan query and that's what I am testing right now. (I am artificially creating that scenario to improve that corner

Re: [PERFORM] Finding bottleneck

2005-08-22 Thread Tom Lane
Merlin Moncure [EMAIL PROTECTED] writes: One thing that might interest you is that the penalty in 8.1 for stats_command_string=true in this type of access pattern is very high: I was experimenting to see if the new cpu efficiency gave me enough of a budget to start using this. This more than

Re: [PERFORM] extremly low memory usage

2005-08-22 Thread Jeremiah Jahn
On Sun, 2005-08-21 at 16:13 -0400, Ron wrote: At 10:54 AM 8/21/2005, Jeremiah Jahn wrote: On Sat, 2005-08-20 at 21:32 -0500, John A Meinel wrote: Ron wrote: Well, since you can get a read of the RAID at 150MB/s, that means that it is actual I/O speed. It may not be cached in RAM.

Re: [PERFORM] extremly low memory usage

2005-08-22 Thread William Yu
Ron wrote: PERC4eDC-PCI Express, 128MB Cache, 2-External Channels Looks like they are using the LSI Logic MegaRAID SCSI 320-2E controller. IIUC, you have 2 of these, each with 2 external channels? A lot of people have mentioned Dell's versions of the LSI cards can be WAY slower than the

Re: [PERFORM] MemoryContextSwitchTo during table scan?

2005-08-22 Thread Tom Lane
Jignesh Shah [EMAIL PROTECTED] writes: Running a script (available on my blog) I find the following top 5 functions where it spends most time during a 10 second run of the script It's pretty risky to draw conclusions from only 10 seconds' worth of gprof data --- that's only 1000 samples total

Re: [PERFORM] MemoryContextSwitchTo during table scan?

2005-08-22 Thread Luke Lonergan
Tom, On 8/22/05 8:41 AM, Tom Lane [EMAIL PROTECTED] wrote: MemoryContextSwitchTo and LockBuffer itself takes 15% of the total time of the query. I was expecting read to be the slowest part (biggest component) but it was way down in the 0.4% level. You do know that gprof counts only CPU

Re: [PERFORM] Finding bottleneck

2005-08-22 Thread Merlin Moncure
That seems quite peculiar; AFAICS the pgstat code shouldn't be any slower than before. At first I thought it might be because we'd increased PGSTAT_ACTIVITY_SIZE, but actually that happened before 8.0 release, so it shouldn't be a factor in this comparison. Just FYI the last time I looked at

Re: [PERFORM] MemoryContextSwitchTo during table scan?

2005-08-22 Thread Josh Berkus
Jignesh, Also is there any way to optimize LockBuffer? Yes, test on 8.1. The buffer manager was re-written for 8.1. You should see a decrease in both LockBuffer and context switch activity. -- --Josh Josh Berkus Aglio Database Solutions San Francisco ---(end of

Re: [PERFORM] MemoryContextSwitchTo during table scan?

2005-08-22 Thread Jignesh Shah
Hi Tom, Like I mentioned I am using DTrace on Solaris 10 x64 and not gprof. DTrace is not based on sampling but actual entry/exit point. Ofcourse my 10 second profile is just a sample that I can assure you is representative of the query since it is a very simple query that does simple table

Re: [PERFORM] MemoryContextSwitchTo during table scan?

2005-08-22 Thread Neil Conway
Jignesh Shah wrote: Now the question is why there are so many calls to MemoryContextSwitchTo in a single SELECT query command? Can it be minimized? I agree with Tom -- if profiling indicates that MemoryContextSwitchTo() is the bottleneck, I would be suspicious that your profiling setup is

Re: [PERFORM] Looking for a large database for testing

2005-08-22 Thread Jim C. Nasby
On Tue, Aug 16, 2005 at 09:29:32AM +0200, Sebastian Hennebrueder wrote: Hello, I would like to test the performance of my Java/PostgreSQL applications especially when making full text searches. For this I am looking for a database with 50 to 300 MB having text fields. e.g. A table with

Re: [PERFORM] Need for speed

2005-08-22 Thread Jim C. Nasby
RRS (http://rrs.decibel.org) might be of use in this case. On Tue, Aug 16, 2005 at 01:59:53PM -0400, Alex Turner wrote: Are you calculating aggregates, and if so, how are you doing it (I ask the question from experience of a similar application where I found that my aggregating PGPLSQL

[PERFORM] pgbench

2005-08-22 Thread Philip Pinkerton
I am looking for the latest pgbench and documentation. If someone know where I can locate them it would save a lot of search time. Thanks Philip Pinkerton TPC-C Benchmarks Sybase Independant Consultant Rio de Janeiro, RJ, Brazil 22031-010 ---(end of