[HACKERS] 500 tpsQL + WAL log implementation

2002-11-15 Thread Curtis Faith
I have been experimenting with empirical tests of file system and device level writes to determine the actual constraints in order to speed up the WAL logging code. Using a raw file partition and a time-based technique for determining the optimal write position, I am able to get 8K writes

Re: [HACKERS] 500 tpsQL + WAL log implementation

2002-11-12 Thread Curtis Faith
tom lane wrote: What can you do *without* using a raw partition? I dislike that idea for two reasons: portability and security. The portability disadvantages are obvious. And in ordinary system setups Postgres would have to run as root in order to write on a raw partition. It occurs to me

Re: [HACKERS] 500 tpsQL + WAL log implementation

2002-11-12 Thread scott.marlowe
Curtis, have you considered comparing raw writes versus file system writes on a raw multi-disk partition? I always set up my machines to store data on a mirror set (RAID1) or RAID5 set, and it seems your method should be tested there too. P.s., Tom, the postgresql user would NOT need to run

Re: [HACKERS] 500 tpsQL + WAL log implementation

2002-11-12 Thread Justin Clift
Bravo Curtis, This is all excellent research. :-) Regards and best wishes, Justin Clift Curtis Faith wrote: snip Disk space is much cheaper than CPU and memory so I think that a logging system that used as much as three or four times the space but is three or four times faster would be a

[HACKERS] 500 tpsQL + WAL log implementation

2002-11-11 Thread Curtis Faith
I have been experimenting with empirical tests of file system and device level writes to determine the actual constraints in order to speed up the WAL logging code. Using a raw file partition and a time-based technique for determining the optimal write position, I am able to get 8K writes

Re: [HACKERS] 500 tpsQL + WAL log implementation

2002-11-11 Thread Tom Lane
Curtis Faith [EMAIL PROTECTED] writes: Using a raw file partition and a time-based technique for determining the optimal write position, I am able to get 8K writes physically written to disk synchronously in the range of 500 to 650 writes per second using FreeBSD raw device partitions on IDE