Re: [HACKERS] OSDL DBT-2 w/ PostgreSQL 7.3.4 and 7.4beta5

2003-11-17 Thread Mark Wong
On Sun, Nov 02, 2003 at 07:25:38AM -0500, Rod Taylor wrote: On Sat, 2003-11-01 at 20:58, Mark Wong wrote: I don't remember making a conscious decision between the number and integer database type. Is that a significant oversight on my part? Numerics do exact math with support

[HACKERS] patching PostgreSQL on STP with dbt-2

2003-12-30 Thread Mark Wong
have any questions. -- Mark Wong - - [EMAIL PROTECTED] Open Source Development Lab Inc - A non-profit corporation 12725 SW Millikan Way - Suite 400 - Beaverton, OR 97005 (503) 626-2455 x 32 (office) (503) 626-2436 (fax) http://developer.osdl.org/markw/ ---(end

Re: [HACKERS] Yet another open-source benchmark

2003-03-03 Thread Mark Wong
://www.postgresql.org/users-lounge/docs/faq.html -- Mark Wong - - [EMAIL PROTECTED] Open Source Development Lab Inc - A non-profit corporation 15275 SW Koll Parkway - Suite H - Beaverton OR, 97006 (503)-626-2455 x 32 (office) (503)-626-2436 (fax) http://www.osdl.org/archive/markw

[HACKERS] some PITR performance data with DBT-2

2004-09-15 Thread Mark Wong
in: 4 x 1.5 GHz Itanium 2 16GB RAM 6 x Compaq Computer Corporation Smart Array 64xx 6 x 14 disk 15K RPM drives (split bus) The database and archive directory were put onto a single LVM volume across all 84 drives. Let me know if I left anything out. -- Mark Wong - - [EMAIL PROTECTED] Open Source

Re: [HACKERS] some PITR performance data with DBT-2

2004-09-15 Thread Mark Wong
On Wed, Sep 15, 2004 at 09:50:17PM +0100, Simon Riggs wrote: Mark Wong wrote Hi Simon, Sorry it has taken so long. Among other things, I doubled the controllers and drives on the system I was testing this on. But now I have some data against PostgreSQL-8.0beta2. Thanks very much

[HACKERS] FunctionCall2 performance

2004-10-04 Thread Mark Wong
is really having that kind of an impact on the test? Links to results are here: 7.5devel - http://www.osdl.org/projects/dbt2dev/results/dev4-010/128/ 8.0beta3 - http://www.osdl.org/projects/dbt2dev/results/dev4-010/180/ Unfortunitely, I lost the database parameters for 7.5devel. :( -- Mark Wong

Re: [HACKERS] FunctionCall2 performance

2004-10-06 Thread Mark Wong
On Mon, Oct 04, 2004 at 02:42:43PM -0400, Greg Stark wrote: Mark Wong [EMAIL PROTECTED] writes: Links to results are here: 7.5devel - http://www.osdl.org/projects/dbt2dev/results/dev4-010/128/ 8.0beta3 - http://www.osdl.org/projects/dbt2dev/results/dev4-010/180/ Are those cyclic spikes

Re: [HACKERS] FunctionCall2 performance

2004-10-07 Thread Mark Wong
On Wed, Oct 06, 2004 at 01:27:26PM -0400, Alvaro Herrera wrote: On Wed, Oct 06, 2004 at 10:01:42AM -0700, Mark Wong wrote: On Mon, Oct 04, 2004 at 02:42:43PM -0400, Greg Stark wrote: Mark Wong [EMAIL PROTECTED] writes: Links to results are here: 7.5devel - http://www.osdl.org

[HACKERS] Call for BOFs Linux World Expo Boston

2004-10-07 Thread Mark Wong
I'd like to set up a BOF with any of the developers who are interested in Boston at the Linux World Expo coming up at the end of February. I don't have anything specific in mind and would like to get something going. Any takers? Some of the things I'm currently working on are OLTP and DSS

Re: [HACKERS] spinlocks: generalizing non-locking test

2004-10-18 Thread Mark Wong
On Sun, Oct 17, 2004 at 11:16:50PM +1000, Neil Conway wrote: Currently, the assembly for TAS() on x86 does a non-locking test before using an atomic operation to attempt to acquire the spinlock: __asm__ __volatile__( cmpb$0,%1 \n jne

Re: [HACKERS] Proposed Query Planner TODO items

2004-10-25 Thread Mark Wong
(send unregister YourEmailAddressHere to [EMAIL PROTECTED]) -- Mark Wong - - [EMAIL PROTECTED] Open Source Development Lab Inc - A non-profit corporation 12725 SW Millikan Way - Suite 400 - Beaverton, OR 97005 (503) 626-2455 x 32 (office) (503) 626-2436 (fax) http://developer.osdl.org/markw

[HACKERS] DBT-3 Query 2 EXPLAIN ANALYZE differences

2004-10-26 Thread Mark Wong
I was doing some testing with DBT-3 on our 8-way STP systems and noticed a significant difference in the execution of Query 2 using 8.0beta3. Here is the query template we're using: select s_acctbal, s_name, n_name, p_partkey, p_mfgr, s_address, s_phone, s_comment from part, supplier,

Re: [HACKERS] ARC Memory Usage analysis

2004-10-27 Thread Mark Wong
On Mon, Oct 25, 2004 at 11:34:25AM -0400, Jan Wieck wrote: On 10/22/2004 4:09 PM, Kenneth Marshall wrote: On Fri, Oct 22, 2004 at 03:35:49PM -0400, Jan Wieck wrote: On 10/22/2004 2:50 PM, Simon Riggs wrote: I've been using the ARC debug options to analyse memory usage on the

Re: [HACKERS] Proposed Query Planner TODO items

2004-10-28 Thread Mark Wong
Hi Tatsuo, I've made a new release: http://prdownloads.sourceforge.net/osdldbt/dbt3-v1.5.tar.gz?download Let me know if there are any problems. On Tue, Oct 26, 2004 at 12:44:49PM +0900, Tatsuo Ishii wrote: Hi, Thanks for the info. Would you give me the tarball?

[HACKERS] 8.0beta5 results w/ dbt2

2004-11-29 Thread Mark Wong
I have some initial results using 8.0beta5 with our OLTP workload. Off the bat I see about a 23% improvement in overall throughput. The most significant thing I've noticed was in the oprofile report where FunctionCall2 and hash_seq_search have moved down the profile a bit. Also, I have libc with

Re: [Testperf-general] Re: [HACKERS] 8.0beta5 results w/ dbt2

2004-11-30 Thread Mark Wong
On Tue, Nov 30, 2004 at 07:12:10AM +, Simon Riggs wrote: If you look at the graph of New Order response time distribution, the higher result gives much more frequent sub-second response for 8.0beta5 and the hump at around 23secs has moved down to 14secs. Notably, the payment transaction

Re: [HACKERS] 8.0beta5 results w/ dbt2

2004-11-30 Thread Mark Wong
On Tue, Nov 30, 2004 at 08:34:20AM +0100, Michael Paesold wrote: Mark Wong wrote: I have some initial results using 8.0beta5 with our OLTP workload. Off the bat I see about a 23% improvement in overall throughput. The most significant thing I've noticed was in the oprofile report where

Re: [HACKERS] 8.0beta5 results w/ dbt2

2004-11-30 Thread Mark Wong
On Tue, Nov 30, 2004 at 10:57:02AM -0500, Tom Lane wrote: Greg Stark [EMAIL PROTECTED] writes: Mark Wong [EMAIL PROTECTED] writes: I have some initial results using 8.0beta5 with our OLTP workload. http://www.osdl.org/projects/dbt2dev/results/dev4-010/199/ throughput: 4076.97 Do

Re: [HACKERS] 8.0beta5 results w/ dbt2

2004-11-30 Thread Mark Wong
On Tue, Nov 30, 2004 at 11:03:03AM -0500, Rod Taylor wrote: On Mon, 2004-11-29 at 16:01 -0800, Mark Wong wrote: I have some initial results using 8.0beta5 with our OLTP workload. Off the bat I see about a 23% improvement in overall throughput. The most significant thing I've noticed

Re: [HACKERS] 8.0beta5 results w/ dbt2

2004-11-30 Thread Mark Wong
On Tue, Nov 30, 2004 at 02:00:29AM -0500, Greg Stark wrote: Mark Wong [EMAIL PROTECTED] writes: I have some initial results using 8.0beta5 with our OLTP workload. http://www.osdl.org/projects/dbt2dev/results/dev4-010/199/ throughput: 4076.97 Do people really only look

Re: [HACKERS] [PATCHES] O_DIRECT for WAL writes

2005-07-06 Thread Mark Wong
On Fri, 24 Jun 2005 09:21:56 -0700 Josh Berkus josh@agliodbs.com wrote: Jim, Josh, is this something that could be done in the performance lab? That's the idea. Sadly, OSDL's hardware has been having critical failures of late (I'm still trying to get test results on the

[HACKERS] dbt2 opteron performance

2005-07-12 Thread Mark Wong
I'm starting to get results with dbt2 on a 4-way opteron system and wanted to share what I've got so far since people have told me in the past that this architecture is more interesting than the itanium2 that I've been using. This 4-way has 8GB of memory and four Adaptec 2200s controllers

Re: [HACKERS] Checkpoint cost, looks like it is WAL/CRC

2005-07-25 Thread Mark Wong
On Fri, 22 Jul 2005 19:11:36 -0400 Tom Lane [EMAIL PROTECTED] wrote: BTW, I'd like to look at 302906, but its [Details] link is broken. Ugh, I tried digging onto the internal systems and it looks like they were destroyed (or not saved) somehow. It'll have to be rerun. Sorry... Mark

Re: [HACKERS] wal_buffer tests in

2005-07-27 Thread Mark Wong
On Wed, 27 Jul 2005 13:30:01 -0700 Josh Berkus josh@agliodbs.com wrote: Folks, I ran a wal_buffer test series. It appears that increasing the wal_buffers is indeed very important for OLTP applications, potentially resulting in as much as a 15% average increase in transaction processing.

Re: [HACKERS] [Testperf-general] dbt2 opteron performance

2005-07-27 Thread Mark Wong
On Wed, Jul 27, 2005 at 07:32:34PM -0700, Josh Berkus wrote: Mark, I'm starting to get results with dbt2 on a 4-way opteron system and wanted to share what I've got so far since people have told me in the past that this architecture is more interesting than the itanium2 that I've been

Re: [HACKERS] [Testperf-general] dbt2 opteron performance

2005-07-28 Thread Mark Wong
On Thu, 28 Jul 2005 17:19:34 -0500 Jim C. Nasby [EMAIL PROTECTED] wrote: On Thu, Jul 28, 2005 at 05:14:41PM -0500, Jim C. Nasby wrote: On Wed, Jul 27, 2005 at 09:31:39PM -0700, Mark Wong wrote: After seeing the discussion about how bad the disk performance is with a lot of scsi

Re: [HACKERS] [Testperf-general] dbt2 opteron performance

2005-07-28 Thread Mark Wong
On Thu, 28 Jul 2005 17:17:25 -0500 Jim C. Nasby [EMAIL PROTECTED] wrote: On Wed, Jul 27, 2005 at 07:32:34PM -0700, Josh Berkus wrote: This 4-way has 8GB of memory and four Adaptec 2200s controllers attached to 80 spindles (eight 10-disk arrays). For those familiar with the schema, here

Re: [HACKERS] [Testperf-general] dbt2 opteron performance

2005-07-28 Thread Mark Wong
On Thu, 28 Jul 2005 18:48:09 -0500 Jim C. Nasby [EMAIL PROTECTED] wrote: On Thu, Jul 28, 2005 at 04:15:31PM -0700, Mark Wong wrote: On Thu, 28 Jul 2005 17:17:25 -0500 Jim C. Nasby [EMAIL PROTECTED] wrote: On Wed, Jul 27, 2005 at 07:32:34PM -0700, Josh Berkus wrote: This 4-way has

Re: [HACKERS] [Testperf-general] dbt2 opteron performance

2005-07-29 Thread Mark Wong
On Fri, 29 Jul 2005 14:39:08 -0500 Jim C. Nasby [EMAIL PROTECTED] wrote: On Thu, Jul 28, 2005 at 05:00:44PM -0700, Mark Wong wrote: On Thu, 28 Jul 2005 16:55:55 -0700 Mark Wong [EMAIL PROTECTED] wrote: On Thu, 28 Jul 2005 18:48:09 -0500 Jim C. Nasby [EMAIL PROTECTED] wrote

Re: [HACKERS] [Testperf-general] dbt2 opteron performance

2005-07-29 Thread Mark Wong
On Fri, 29 Jul 2005 14:57:42 -0500 Jim C. Nasby [EMAIL PROTECTED] wrote: On Fri, Jul 29, 2005 at 12:51:57PM -0700, Mark Wong wrote: Not sure I fully understand what you're trying to say, but it seems like it might still be worth trying my original idea of just turning all 80 disks

Re: [HACKERS] [Testperf-general] dbt2 opteron performance

2005-07-29 Thread Mark Wong
On Fri, 29 Jul 2005 13:35:32 -0700 Josh Berkus josh@agliodbs.com wrote: Mark, I have done that before actually, when the tablespace patch came out. I was able to get almost 40% more throughput with half the drives than striping all the disks together. That's not the figures you

Re: [HACKERS] [Testperf-general] dbt2 opteron performance

2005-07-29 Thread Mark Wong
On Fri, 29 Jul 2005 13:19:06 -0700 Luke Lonergan [EMAIL PROTECTED] wrote: Mark, On 7/29/05 12:51 PM, Mark Wong [EMAIL PROTECTED] wrote: Adaptec 2200s Have you tried non-RAID SCSI controllers in this configuration? When we used the Adaptec 2120s previously, we got very poor

[HACKERS] dbt3 data with 10GB scale factor

2005-08-02 Thread Mark Wong
I've started scaling dbt3 up to the 10GB scale factor against CVS and the fast COPY patch: http://www.testing.osdl.org/projects/dbt3testing/results/dev4-010/53/ I'm sure there are some better database parameters I should use so please let me know what to try. ;) What I've found interesting is

Re: [HACKERS] data on devel code perf dip

2005-08-12 Thread Mark Wong
On Thu, 11 Aug 2005 22:11:42 -0400 (EDT) Bruce Momjian pgman@candle.pha.pa.us wrote: Tom Lane wrote: Bruce Momjian pgman@candle.pha.pa.us writes: O_DIRECT is only being used for WAL page writes (or I sure hope so anyway), so shared_buffers should be irrelevant. Uh, O_DIRECT really

Re: [HACKERS] data on devel code perf dip

2005-08-16 Thread Mark Wong
On Tue, 16 Aug 2005 18:53:55 -0400 Tom Lane [EMAIL PROTECTED] wrote: Mary Edie Meredith [EMAIL PROTECTED] writes: I'm still very concerned about what I'm seeing in the oprofile: namely: .CreateLWLocks is the second highest entry for postgres.

Re: [HACKERS] Simplifying wal_sync_method

2005-08-29 Thread Mark Wong
On Sun, 21 Aug 2005 19:27:35 -0500 Jim C. Nasby [EMAIL PROTECTED] wrote: On Mon, Aug 08, 2005 at 07:45:38PM -0400, Andrew Dunstan wrote: So the short answer is possibly You build the tests and we'll run 'em. Would some version of dbt2/3 work for this? Yeah, trying... On the larger system

[HACKERS] dbt-4 (tpc-app) kit

2005-09-02 Thread Mark Wong
: http://prdownloads.sourceforge.net/osdldbt/dbt4-0.2.tar.gz?download I don't think my Java skills are very good, so if anyone would like to help, let me know. -- Mark Wong - - [EMAIL PROTECTED] Open Source Development Lab Inc - A non-profit corporation 12725 SW Millikan Way - Suite 400 - Beaverton

Re: [HACKERS] dbt-4 (tpc-app) kit

2005-09-02 Thread Mark Wong
be reviewed too. ;) Mark On Fri, 2 Sep 2005 15:17:46 -0400 Dave Cramer [EMAIL PROTECTED] wrote: Mark, I'd like to help out, let me know what you need help doing. Personally, doing this with EJB's is distasteful, but I'll help where I can. Dave On 2-Sep-05, at 12:53 PM, Mark Wong wrote

Re: [HACKERS] dbt-4 (tpc-app) kit

2005-09-06 Thread Mark Wong
On Fri, 2 Sep 2005 15:50:25 -0400 Dave Cramer [EMAIL PROTECTED] wrote: On 2-Sep-05, at 3:38 PM, Mark Wong wrote: Hi Dave, Oops, EJB's are distasteful? My experience in this area is quite lacking. Well, I said personally distasteful. Not that I necessarily want to be 100% strict

[HACKERS] postgresql CVS callgraph data from dbt2

2005-09-13 Thread Mark Wong
Hi everyone, For those of you watching the the daily results generated from STP (http://developer.osdl.org/markw/postgrescvs/dbt2/) I have callgraph data from oprofile collected starting from the Sept 9 results. Here is an example of what it looks like:

Re: [HACKERS] Spinlocks, yet again: analysis and proposed patches

2005-09-13 Thread Mark Wong
On Tue, 13 Sep 2005 12:21:45 -0400 Douglas McNaught [EMAIL PROTECTED] wrote: Josh Berkus josh@agliodbs.com writes: Tom, All: It seems to me what you've found is an outright bug in the linux scheduler. Perhaps posting it to linux-kernel would be worthwhile. For people using this on

Re: [HACKERS] Spinlocks, yet again: analysis and proposed patches

2005-10-13 Thread Mark Wong
On Wed, 12 Oct 2005 18:44:50 +0100 Simon Riggs [EMAIL PROTECTED] wrote: On Wed, 2005-09-14 at 13:32 -0400, Tom Lane wrote: I wrote: Another thought came to mind: maybe the current data layout for LWLocks is bad. Right now, the spinlock that protects each LWLock data struct is itself

Re: [HACKERS] Spinlocks, yet again: analysis and proposed patches

2005-10-31 Thread Mark Wong
On Thu, 20 Oct 2005 23:03:47 +0100 Simon Riggs [EMAIL PROTECTED] wrote: On Wed, 2005-10-19 at 14:07 -0700, Mark Wong wrote: This isn't exactly elegant coding, but it provides a useful improvement on an 8-way SMP box when run on 8.0 base. OK, lets be brutal: this looks pretty darn

Re: [HACKERS] Spinlocks, yet again: analysis and proposed patches

2005-11-03 Thread Mark Wong
On Tue, 01 Nov 2005 07:32:32 + Simon Riggs [EMAIL PROTECTED] wrote: On Mon, 2005-10-31 at 16:10 -0800, Mark Wong wrote: On Thu, 20 Oct 2005 23:03:47 +0100 Simon Riggs [EMAIL PROTECTED] wrote: On Wed, 2005-10-19 at 14:07 -0700, Mark Wong wrote: This isn't exactly elegant

Re: [HACKERS] Spinlocks, yet again: analysis and proposed patches

2005-11-04 Thread Mark Wong
On Thu, 03 Nov 2005 18:29:09 + Simon Riggs [EMAIL PROTECTED] wrote: On Thu, 2005-11-03 at 08:03 -0800, Mark Wong wrote: On Tue, 01 Nov 2005 07:32:32 + Simon Riggs [EMAIL PROTECTED] wrote: Concerned about the awful checkpointing. Can you bump wal_buffers to 8192 just to make

Re: [HACKERS] Need access to a Linux box

2004-12-06 Thread Mark Wong
OSDL provides hardware. You have to sign up for a login here: https://www.osdl.org/join_form And submit a project proposal here: https://www.osdl.org/lab_activities/lab_projects/a_propose_project/propose_a_project.html Mark ---(end of

Re: [Testperf-general] Re: [HACKERS] 8.0beta5 results w/ dbt2

2004-12-06 Thread Mark Wong
On Tue, Nov 30, 2004 at 10:51:42PM +, Simon Riggs wrote: My suggestion: increase checkpoint_timeout to 600 secs, increase bgwriter parameters also, to reduce how frequently it is called, as well as increase the number of blocks per cycle. Ok, here are a series of three tests varying the

Re: [Testperf-general] Re: [HACKERS] 8.0beta5 results w/ dbt2

2004-12-06 Thread Mark Wong
On Mon, Dec 06, 2004 at 09:28:15PM +, Simon Riggs wrote: Mark, Few questions: - can we put the logging to DEBUG1 please, so we can see the checkpoints? ...and set debug_shared_buffers = 10 Ok, will do. I don't understand why the checkpoints are so regular at 300 seconds if the

Re: [Testperf-general] Re: [HACKERS] 8.0beta5 results w/ dbt2

2004-12-06 Thread Mark Wong
On Mon, Dec 06, 2004 at 11:44:22PM +, Simon Riggs wrote: On the graphs... why do the graphs for Proc Utilisation, Index Scans etc, only show first 300 secs of a 3600 sec long run? Are those axes correct? (I understand seeing the ramp-up is important, I just want to check the time axis).

Re: [Testperf-general] Re: [HACKERS] 8.0beta5 results w/ dbt2

2004-12-10 Thread Mark Wong
On Mon, Dec 06, 2004 at 07:52:37PM +, Simon Riggs wrote: Varying bgwriter_maxpages upwards should take performance higher. I have 2 runs now. I for both tests, I have bgwriter_percent=100, checkpoint_segments=8192, checkpoint_timout=600, debug_shared_buffers=10, log_min_messages=debug1

Re: [Testperf-general] Re: [HACKERS] 8.0beta5 results w/ dbt2

2004-12-10 Thread Mark Wong
On Tue, Dec 07, 2004 at 09:12:18AM +, Simon Riggs wrote: Not sure, as yet, what is causing effect 2. It's not related to the kernel, but is related to user CPU and I/O waits and effects all tables in proportion to their overall I/O usage. Some evidence that it becomes more pronounced as

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-14 Thread Mark Wong
Sorry for the delay; here are results with the bg3.patch with database parameters that should match run 207. I haven't been able to take the time too look over the results myself, but I tried to make sure this run was the same as 207:

Re: [HACKERS] [Testperf-general] BufferSync and bgwriter

2004-12-14 Thread Mark Wong
Sorry, wrong link, right one here: http://www.osdl.org/projects/dbt2dev/results/dev4-010/211 Mark ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org

[HACKERS] pg_autovacuum w/ dbt2

2004-12-21 Thread Mark Wong
After all this time I finally got around to vacuuming the database with dbt2 with pg_autovacuum. :) http://www.osdl.org/projects/dbt2dev/results/dev4-010/215/ Doesn't look so good though, probably because I'm not using optimal settings with pg_autovacuum. So far I have only tried the

Re: [HACKERS] pg_autovacuum w/ dbt2

2004-12-21 Thread Mark Wong
On Tue, Dec 21, 2004 at 02:23:41PM -0500, Tom Lane wrote: Mark Wong [EMAIL PROTECTED] writes: [2004-12-20 15:48:18 PST] The error is [ERROR: failed to re-find parent key in pk_district ] Yikes. Is this reproducible? regards, tom lane Yes, and I think

Re: [HACKERS] pg_autovacuum w/ dbt2

2004-12-21 Thread Mark Wong
changing the other settings, if that sounds good. Mark On Tue, Dec 21, 2004 at 02:33:57PM -0500, Matthew T. O'Connor wrote: Mark Wong wrote: After all this time I finally got around to vacuuming the database with dbt2 with pg_autovacuum. :) http://www.osdl.org/projects/dbt2dev/results

Re: [HACKERS] pg_autovacuum w/ dbt2

2004-12-21 Thread Mark Wong
On Tue, Dec 21, 2004 at 05:56:47PM -0500, Tom Lane wrote: Mark Wong [EMAIL PROTECTED] writes: On Tue, Dec 21, 2004 at 02:23:41PM -0500, Tom Lane wrote: Mark Wong [EMAIL PROTECTED] writes: [2004-12-20 15:48:18 PST] The error is [ERROR: failed to re-find parent key in pk_district

Re: [HACKERS] [Testperf-general] pg_autovacuum w/ dbt2

2005-01-06 Thread Mark Wong
with the errors, so I'll continue with trying to tune some of the pg_autovacuum values. Mark On Tue, Dec 21, 2004 at 09:41:31AM -0800, Mark Wong wrote: After all this time I finally got around to vacuuming the database with dbt2 with pg_autovacuum. :) http://www.osdl.org/projects/dbt2dev

Re: [HACKERS] [Testperf-general] pg_autovacuum w/ dbt2

2005-01-07 Thread Mark Wong
On Fri, Jan 07, 2005 at 09:58:47AM -0800, Josh Berkus wrote: Mark, No manual vacuum commands before. The decline in performance has been pretty consistent in all my previous tests and people have told me on many occasions that the decline in performance was probably because I was never

Re: [HACKERS] [Testperf-general] pg_autovacuum w/ dbt2

2005-01-07 Thread Mark Wong
On Fri, Jan 07, 2005 at 10:13:52AM -0800, Josh Berkus wrote: Mark, All default parameters. Matthew also recommended using the vacuum_delay setting so I was about to try that. Interesting ... the default parameters are quite conservative, running only when the table has doubled in new

[HACKERS] sparse (static analyzer) report

2005-01-12 Thread Mark Wong
Hi, Just wondering if anyone finds spare's analysis useful. I ran it against 8.0-rc5: http://developer.osdl.org/markw/pgsql/sparse/pg-8.0rc5.txt Sparse can be downloaded http://www.codemonkey.org.uk/projects/bitkeeper/sparse/ or bk://sparse.bkbits.net/sparse

Re: [HACKERS] pg_autovacuum w/ dbt2

2005-01-12 Thread Mark Wong
On Tue, Dec 21, 2004 at 05:56:47PM -0500, Tom Lane wrote: If you want to track it yourself, please change those elog(ERROR)s to elog(PANIC) so that they'll generate core dumps, then build with --enable-debug if you didn't already (--enable-cassert would be good too) and get a debugger stack

Re: [HACKERS] pg_autovacuum w/ dbt2

2005-01-12 Thread Mark Wong
? Mark /* * This file is released under the terms of the Artistic License. Please see * the file LICENSE, included in this package, for details. * * Copyright (C) 2003 Mark Wong Open Source Development Lab, Inc. * * Based on TPC-C Standard Specification Revision 5.0 Clause 2.8.2

Re: [HACKERS] sparse (static analyzer) report

2005-01-13 Thread Mark Wong
We've also started automating sparse analyses in our PLM tool, which will show an error and warning count. Here's an example: http://www.osdl.org/plm-cgi/plm?module=patch_infopatch_id=4065 On Wed, Jan 12, 2005 at 02:18:36PM -0800, Mark Wong wrote: Hi, Just wondering if anyone finds

Re: [HACKERS] sparse (static analyzer) report

2005-01-13 Thread Mark Wong
On Thu, Jan 13, 2005 at 08:06:23PM -0300, Alvaro Herrera wrote: On Thu, Jan 13, 2005 at 01:31:36PM -0800, Mark Wong wrote: We've also started automating sparse analyses in our PLM tool, which will show an error and warning count. Here's an example: http://www.osdl.org/plm-cgi/plm

Re: [HACKERS] sparse (static analyzer) report

2005-01-14 Thread Mark Wong
On Thu, Jan 13, 2005 at 03:09:19PM -0800, Mark Wong wrote: On Thu, Jan 13, 2005 at 08:06:23PM -0300, Alvaro Herrera wrote: On Thu, Jan 13, 2005 at 01:31:36PM -0800, Mark Wong wrote: We've also started automating sparse analyses in our PLM tool, which will show an error and warning count

Re: [HACKERS] sparse (static analyzer) report

2005-01-14 Thread Mark Wong
On Fri, Jan 14, 2005 at 09:54:24PM -0300, Alvaro Herrera wrote: On Fri, Jan 14, 2005 at 03:53:09PM -0800, Mark Wong wrote: On Thu, Jan 13, 2005 at 03:09:19PM -0800, Mark Wong wrote: On Thu, Jan 13, 2005 at 08:06:23PM -0300, Alvaro Herrera wrote: On Thu, Jan 13, 2005 at 01:31:36PM -0800

Re: [HACKERS] sparse (static analyzer) report

2005-01-14 Thread Mark Wong
Ah, so you beat me to it Neil. ;) Out of curiosity, how much worse was it before you started fixing things? Mark On Sat, Jan 15, 2005 at 01:30:37PM +1100, Neil Conway wrote: BTW, perhaps one reason for the relatively small number of legitimate issues picked up by sparse is that I ran sparse

[HACKERS] OLS BOF for linux postgresql

2005-01-25 Thread Mark Wong
I wanted to bounce the idea of a BOF at the Linux Symposium in Ottawa and see if anyone would like to attend. The deadline to proposal is Feb 1st, sort of short notice... I thought the dicussion could revolved around these two topics: Linux features that PostgreSQL should take advantage of.

Re: [HACKERS] WAL: O_DIRECT and multipage-writer

2005-01-27 Thread Mark Wong
Hi everyone, I gave this a try with DBT-2, but got a core dump on our ia64 system. I hope this isn't a random thing, like I ran into previously. Maybe I'll try again, but postgres dumped core. Binary and core here: http://developer.osdl.org/markw/pgsql/core/2morefiles.tar.bz2 #0

Re: [PATCHES] [HACKERS] WAL: O_DIRECT and multipage-writer

2005-01-27 Thread Mark Wong
Hmm... I don't remember specifying a datatype. I suppose whatever the default one is. :) I'll be happy to test again, just let me know. Mark On Fri, Jan 28, 2005 at 06:28:32AM +0900, ITAGAKI Takahiro wrote: Thanks for testing, Mark! I gave this a try with DBT-2, but got a core dump on our

Re: [HACKERS] 8.0.X and the ARC patent

2005-03-01 Thread Mark Wong
On Sat, Feb 19, 2005 at 12:57:52AM -0500, Tom Lane wrote: So far I've not been able to measure any consistent difference, but you know how much I trust pgbench ;-). I hope that Mark Wong can give us some results on the OSDL setup soon. Sorry for the delay, broken laptop, vacation, etc

Re: [PATCHES] [HACKERS] WAL: O_DIRECT and multipage-writer (+ memory leak)

2005-03-01 Thread Mark Wong
On Thu, Feb 03, 2005 at 07:25:55PM +0900, ITAGAKI Takahiro wrote: Hello everyone. I fixed two bugs in the patch that I sent before. Check and test new one, please. Ok, finally got back into the office and was able to run 1 set of tests. So the new baseline result with 8.0.1:

Re: [HACKERS] 8.0.X and the ARC patent

2005-03-01 Thread Mark Wong
On Tue, Mar 01, 2005 at 04:57:11PM -0500, Tom Lane wrote: Mark Wong [EMAIL PROTECTED] writes: About a 6% increase, but if you look at the performance over time, it's degrading steadily. The latter throughput peaks near 5000. Curious. The immediate question is does it ever flatten out

Re: [HACKERS] 8.0.X and the ARC patent

2005-03-02 Thread Mark Wong
On Tue, Mar 01, 2005 at 05:17:07PM -0500, Tom Lane wrote: Mark Wong [EMAIL PROTECTED] writes: On Tue, Mar 01, 2005 at 04:57:11PM -0500, Tom Lane wrote: Curious. The immediate question is does it ever flatten out, and if so at what TPM rate compared to 8.0.1? Could you run the same test

Re: [HACKERS] 8.0.X and the ARC patent

2005-03-02 Thread Mark Wong
wrote: I was just looking at the config parameters, and you have the shared buffers set to 60k, and the effective cache set to 1k Dave Mark Wong wrote: On Tue, Mar 01, 2005 at 05:17:07PM -0500, Tom Lane wrote: Mark Wong [EMAIL PROTECTED] writes: On Tue, Mar 01, 2005

Re: [HACKERS] 8.0.X and the ARC patent

2005-03-02 Thread Mark Wong
On Wed, Mar 02, 2005 at 03:15:54PM -0500, Tom Lane wrote: Mark Wong [EMAIL PROTECTED] writes: Ah, ok. I've reapplied the 2Q patch to CVS from 20050301: http://www.osdl.org/projects/dbt2dev/results/dev4-010/313/ I ran it for 3 hours, just in case, and the charts suggest it flattens

Re: [HACKERS] 8.0.X and the ARC patent

2005-03-02 Thread Mark Wong
the table it appears that you are changing vm parameters as well as database configuration parameters between runs ? Dave Mark Wong wrote: Yes, those parameters are based on a series of test results here: http://www.osdl.org/projects/dbt2dev/results/pgsql/rc4.html Run 264 provided

Re: [HACKERS] WAL: O_DIRECT and multipage-writer

2005-03-22 Thread Mark Wong
On Tue, Jan 25, 2005 at 06:06:23PM +0900, ITAGAKI Takahiro wrote: Environment: OS : Linux kernel 2.6.9 CPU: Pentium 4 3GHz disk : ATA 5400rpm (Data and WAL are placed on same partition.) memory : 1GB config : shared_buffers=1, wal_buffers=256,

Re: [HACKERS] WAL: O_DIRECT and multipage-writer

2005-03-23 Thread Mark Wong
On Wed, Mar 23, 2005 at 01:55:46PM +0900, ITAGAKI Takahiro wrote: Hi, Mark. Mark Wong [EMAIL PROTECTED] wrote: In light of this thread, have you compared the performance on Linux-2.4? No, but I'm just testing my patch on Linux-2.4 with a middle-range server. I will report the results

[HACKERS] PLM pulling from CVS nightly for testing in STP

2005-04-13 Thread Mark Wong
Hi all, Just wanted everyone to know what we're pulling CVS HEAD nightly so it can be tested in STP now. Let me know if you have any questions. Tests are not automatically run yet, but I hope to remedy that shortly. For those not familiar with STP and PLM, here are a couple of links: STP

Re: [HACKERS] PLM pulling from CVS nightly for testing in STP

2005-04-13 Thread Mark Wong
On Wed, Apr 13, 2005 at 11:35:36AM -0700, Josh Berkus wrote: Mark, Just wanted everyone to know what we're pulling CVS HEAD nightly so it can be tested in STP now. Let me know if you have any questions. Way cool.How do I find the PLM number? How are you nameing these? The naming

Re: [HACKERS] PLM pulling from CVS nightly for testing in STP

2005-04-19 Thread Mark Wong
I have dbt-2 tests automatically running against each pull from CVS and have started to automatically compile results here: http://developer.osdl.org/markw/postgrescvs/ I did start with a bit of a minimalistic approach, so I'm open for any comments, feedback, etc. Mark

[HACKERS] Daily DBT-3 (DSS) Results on CVS head

2005-06-07 Thread Mark Wong
FYI, I have results being generated daily against CVS, in addition to dbt2: http://developer.osdl.org/markw/postgrescvs/ I've also added a link to instructions on how to submit patches to test against PostgreSQL on that page, if anyone's interested. Thanks, Mark

Re: [HACKERS] Daily DBT-3 (DSS) Results on CVS head

2005-06-10 Thread Mark Wong
On Fri, 10 Jun 2005 19:17:58 +0900 Junji TERAMOTO [EMAIL PROTECTED] wrote: Hello Mark, Mark Wong wrote: http://developer.osdl.org/markw/postgrescvs/ This site includes Profile Report. It's very interesting and useful! Then, I compared result of 5/29 with the result of 5/30

[HACKERS] Separate BLCKSZ for data and logging

2006-03-16 Thread Mark Wong
Hi all, I've been wondering if there might be anything to gain by having a separate block size for logging and data. I thought I might try defining DATA_BLCKSZ and LOG_BLCKSZ and see what kind of trouble I get myself into. I wasn't able to find any previous discussion but pehaps 'separate

Re: [HACKERS] Separate BLCKSZ for data and logging

2006-03-16 Thread Mark Wong
On Thu, 16 Mar 2006 19:37:07 + Simon Riggs [EMAIL PROTECTED] wrote: On Thu, 2006-03-16 at 08:21 -0800, Mark Wong wrote: I've been wondering if there might be anything to gain by having a separate block size for logging and data. I thought I might try defining DATA_BLCKSZ

Re: [HACKERS] Separate BLCKSZ for data and logging

2006-03-16 Thread Mark Wong
On Thu, 16 Mar 2006 20:51:54 + Simon Riggs [EMAIL PROTECTED] wrote: On Thu, 2006-03-16 at 12:22 -0800, Mark Wong wrote: I was hoping that in the case where 2 or more data blocks are written to the log that they could written once within a single larger log block. The log block size

Re: [HACKERS] Patch Submission Guidelines

2006-03-22 Thread Mark Wong
On Tue, 14 Feb 2006 21:54:12 + Simon Riggs [EMAIL PROTECTED] wrote: On Tue, 2006-02-14 at 16:17 -0500, Andrew Dunstan wrote: If I had enough time there are all sorts of things like this I'd love to set up. A fetchable url that says try these experimental CVS branches or something

Re: [HACKERS] Summer of Code

2006-04-18 Thread Mark Wong
Josh Berkus wrote: Jonah, Where do we stand on this? Google sent me the docs on this year's SoC literally 2 hours ago. I need to read through them and start trying to connect mentors and students and projects. Do you think a proposal to work on a TPC-App (Java) and TPC-E (next

Re: [HACKERS] Google SoC--Idea Request

2006-04-18 Thread Mark Wong
Jim C. Nasby wrote: On Sat, Apr 15, 2006 at 03:05:20PM -0400, Jonah H. Harris wrote: All ideas welcome! I know it's not directly PostgreSQL related, but I'd love to see the dbt* code improved. Items on my wish-list: - make it easy to run the test framework and clients on a seperate machine

Re: [HACKERS] Google SoC--Idea Request

2006-04-18 Thread Mark Wong
Jonah H. Harris wrote: On 4/18/06, Jim C. Nasby [EMAIL PROTECTED] wrote: On Tue, Apr 18, 2006 at 11:27:40AM -0700, Mark Wong wrote: Jim C. Nasby wrote: On Sat, Apr 15, 2006 at 03:05:20PM -0400, Jonah H. Harris wrote: All ideas welcome! I know it's not directly PostgreSQL related, but I'd

Re: [HACKERS] Further reduction of bufmgr lock contention

2006-04-24 Thread Mark Wong
Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: On Fri, 2006-04-21 at 13:01 -0400, Tom Lane wrote: I've been looking into Gavin Hamill's recent report of poor performance with PG 8.1 on an 8-way IBM PPC64 box. Keep in mind that Gavin's 8-way turns back into a pumpkin on Monday :-( I

[HACKERS] XLOG_BLCKSZ vs. wal_buffers table

2006-04-30 Thread Mark Wong
I would have gotten this out sooner but I'm having trouble with our infrastructure. Here's a link to a table of data I've started putting together regarding XLOG_BLCKSZ and wal_buffers on a 4-way Opteron system: http://developer.osdl.org/markw/pgsql/xlog_blcksz.html There are a couple of

Re: [HACKERS] XLOG_BLCKSZ vs. wal_buffers table

2006-05-05 Thread Mark Wong
On Tue, 02 May 2006 10:52:38 +0100 Simon Riggs [EMAIL PROTECTED] wrote: On Sun, 2006-04-30 at 22:14 -0700, Mark Wong wrote: I would have gotten this out sooner but I'm having trouble with our infrastructure. Here's a link to a table of data I've started putting together regarding

Re: [HACKERS] XLOG_BLCKSZ vs. wal_buffers table

2006-05-08 Thread Mark Wong
On Mon, 08 May 2006 19:08:59 +0100 Simon Riggs [EMAIL PROTECTED] wrote: On Fri, 2006-05-05 at 16:00 -0700, Mark Wong wrote: On Tue, 02 May 2006 10:52:38 +0100 Simon Riggs [EMAIL PROTECTED] wrote: On Sun, 2006-04-30 at 22:14 -0700, Mark Wong wrote: I would have gotten this out

Re: [HACKERS] XLOG_BLCKSZ vs. wal_buffers table

2006-05-10 Thread Mark Wong
Simon Riggs wrote: Could you turn full_page_writes = off and do a few more tests? I think the full page writes is swamping the xlog and masking the performance we might see for normal small xlog writes. I'd try XLOG_BLCKSZ = 4096 and 8192 to start with. Thanks. Ok, got data for XLOG_BLCKXZ at

Re: [HACKERS] XLOG_BLCKSZ vs. wal_buffers table

2006-05-11 Thread Mark Wong
Simon Riggs wrote: On Wed, 2006-05-10 at 09:55 -0700, Mark Wong wrote: Simon Riggs wrote: Could you turn full_page_writes = off and do a few more tests? I think the full page writes is swamping the xlog and masking the performance we might see for normal small xlog writes. I'd try XLOG_BLCKSZ

Re: [HACKERS] OSDL DBT-2 for PostgreSQL

2003-08-01 Thread Mark Wong
On Fri, Aug 01, 2003 at 05:05:18PM -0700, Josh Berkus wrote: Mark, I've just got our DBT-2 workload (TPC-C derivate) working with PostgreSQL using C stored functions and libpq. I'd love to get some feedback. I'm confused. Jenny Zhang just announced OSDL-DBT3 for Postgres; is this a

Re: [HACKERS] OSDL DBT-2 w/ PostgreSQL 7.3.4 and 7.4beta5

2003-11-01 Thread Mark Wong
On Sat, Nov 01, 2003 at 02:37:21PM +0100, Manfred Spraul wrote: Tom Lane wrote: [EMAIL PROTECTED] writes: 7.4beta5 offers more throughput. One significant difference I see is in the oprofile for the database. For the additional 7% increase in the metric, there are about 32% less

  1   2   >