Re: [HACKERS] Win32 Powerfail testing

2003-03-07 Thread Kevin Brown
Tatsuo Ishii wrote: Today I revisited the implemnetation (replacing sync() with open/_commit) I made several days ago and found a bug with it (thanks to Hiroshi). With the fixed version of it, now my Win32 port has passed your test even right after checkpoint!. I presume that this

Re: [HACKERS] Win32 Powerfail testing

2003-03-07 Thread Kevin Brown
Bruce Momjian wrote: The idea of using this on Unix is tempting, but Tatsuo is using a threaded backend, so it is a little easier to do. However, it would probably be pretty easy to write a file of modified file names that the checkpoint could read and open/fsync/close. Even that's not

Re: [HACKERS] Win32 Powerfail testing

2003-03-07 Thread Kevin Brown
Bruce Momjian wrote: Kevin Brown wrote: Bruce Momjian wrote: The idea of using this on Unix is tempting, but Tatsuo is using a threaded backend, so it is a little easier to do. However, it would probably be pretty easy to write a file of modified file names that the checkpoint

Re: [HACKERS] Win32 Powerfail testing

2003-03-07 Thread Tatsuo Ishii
But even then, we don't actually have to track the *names* of the files that have changed, just their RelFileNodes, since there's a mapping function from the RelFileNode to the filename. Right. I have noticed that too and have made changes to my implementaion. BTW, you need to track the block

Re: [HACKERS] Win32 Powerfail testing

2003-03-07 Thread Greg Stark
Kevin Brown [EMAIL PROTECTED] writes: Even that's not strictly necessary -- we *do* have shared memory we can use for this, and even when hundreds of tables have been written the list will only end up being a few tens of kilobytes in size (plus whatever overhead is required to track and

Re: [HACKERS] Win32 Powerfail testing

2003-03-07 Thread Bruce Momjian
Kevin Brown wrote: Bruce Momjian wrote: The idea of using this on Unix is tempting, but Tatsuo is using a threaded backend, so it is a little easier to do. However, it would probably be pretty easy to write a file of modified file names that the checkpoint could read and

Re: [HACKERS] Win32 Powerfail testing

2003-03-07 Thread Bruce Momjian
Kevin Brown wrote: Tatsuo Ishii wrote: Today I revisited the implemnetation (replacing sync() with open/_commit) I made several days ago and found a bug with it (thanks to Hiroshi). With the fixed version of it, now my Win32 port has passed your test even right after checkpoint!. I

Re: [HACKERS] Win32 Powerfail testing

2003-03-06 Thread Tatsuo Ishii
It would be an interesting comparison for you to roll the file descriptor tracking changes into the Unix side of the tree and use fsync() or fdatasync() in place of FlushFileBuffers() on the Unix side (you'd have to remove or disable the code that does a sync() of course). If the end result

Re: [HACKERS] Win32 Powerfail testing

2003-03-06 Thread Dave Page
-Original Message- From: Kevin Brown [mailto:[EMAIL PROTECTED] Sent: 06 March 2003 04:37 To: [EMAIL PROTECTED] Subject: Re: [HACKERS] Win32 Powerfail testing Tatsuo Ishii wrote: Sorry, but it does not help. The page says we could use FlushFileBuffers() to sync the kernel

Re: [HACKERS] Win32 Powerfail testing

2003-03-06 Thread Dave Page
-Original Message- From: Tatsuo Ishii [mailto:[EMAIL PROTECTED] Sent: 05 March 2003 13:49 To: Dave Page Cc: [EMAIL PROTECTED] Subject: Re: [HACKERS] Win32 Powerfail testing So far we found interesting facts. Our Win32 port passes his test in most cases. However if power

Re: [HACKERS] Win32 Powerfail testing

2003-03-06 Thread Merlin Moncure
-Original Message- From: Dave Page [mailto:[EMAIL PROTECTED] Sent: Thursday, March 06, 2003 11:02 AM To: Tatsuo Ishii Cc: [EMAIL PROTECTED] Subject: Re: [HACKERS] Win32 Powerfail testing -Original Message- From: Tatsuo Ishii [mailto:[EMAIL PROTECTED] Sent: 06 March 2003 15

Re: [HACKERS] Win32 Powerfail testing

2003-03-06 Thread Tatsuo Ishii
Sorry, but it does not help. The page says we could use FlushFileBuffers() to sync the kernel buffer to the disk. Unfortunately, it requires a file descriptor to flush for its argument. Thus it could not be a replacement of sync(). Actually I have modified the buffer manager so that

Re: [HACKERS] Win32 Powerfail testing

2003-03-06 Thread Dave Page
-Original Message- From: Tatsuo Ishii [mailto:[EMAIL PROTECTED] Sent: 06 March 2003 14:00 To: Dave Page Cc: [EMAIL PROTECTED] Subject: Re: [HACKERS] Win32 Powerfail testing Sorry, but it does not help. The page says we could use FlushFileBuffers() to sync the kernel

Re: [HACKERS] Win32 Powerfail testing

2003-03-06 Thread Tatsuo Ishii
As I said in the previlus mails, open()+_commit() does the right job with the transaction log files. So probably I think I should stick with open()+_commit() approach for ordinary table/index files too. Oh, I didn't see that message. So it's either: open() + _commit() Sorry, I

Re: [HACKERS] Win32 Powerfail testing

2003-03-06 Thread Magnus Hagander
Agreed, but I still keep thinking that despite some peoples claims that Windows ain't up to it, DB2, SQL and Exchange Server as well a probably others that don't use raw partitions have got over this problem, so therefore we should be able to. Admittedly Microsoft have a bit of an

Re: [HACKERS] Win32 Powerfail testing

2003-03-06 Thread Tatsuo Ishii
Are you asking the way how to open files in the buffer manager? If so, basically PostgreSQL uses open() with flags (O_RDWR | PG_BINARY, 0600). I cannot find it now, but I'm sure I read that FlushFileBuffers() has no effect unless the file was opened with CreateFile() with the

[HACKERS] Win32 Powerfail testing

2003-03-05 Thread Tatsuo Ishii
We are developing a Win32 port of PostgreSQL 7.3(different from Jan's implementaion, in that we are using a thread model. In the future I hope we could contribute the source code). We have done a power failure testing using the test tool made by Dave Page: Subject: [HACKERS] Win32 Powerfail

Re: [HACKERS] Win32 Powerfail testing

2003-03-05 Thread Kevin Brown
Tatsuo Ishii wrote: Sorry, but it does not help. The page says we could use FlushFileBuffers() to sync the kernel buffer to the disk. Unfortunately, it requires a file descriptor to flush for its argument. Thus it could not be a replacement of sync(). Actually I have modified the buffer

Re: [HACKERS] Win32 Powerfail testing

2003-03-05 Thread scott.marlowe
On Wed, 5 Mar 2003, Dave Page wrote: -Original Message- From: Tatsuo Ishii [mailto:[EMAIL PROTECTED] Sent: 05 March 2003 02:23 To: [EMAIL PROTECTED] Subject: [HACKERS] Win32 Powerfail testing So far we found interesting facts. Our Win32 port passes his test in most

Re: [HACKERS] Win32 Powerfail testing - results

2003-02-03 Thread Dave Page
Vince Vielhaber allegedly said: On Mon, 3 Feb 2003, Dave Page wrote: Run | Errors Detected = 07 | COUNT CHECK - Duplicate or missing rows detected (10262)!! 09 | DISTINCT CHECK - Duplicate or missing rows detected (9893)!! |

Re: [HACKERS] Win32 Powerfail testing - results

2003-02-03 Thread Dave Page
-Original Message- From: Tom Lane [mailto:[EMAIL PROTECTED]] Sent: 03 February 2003 21:52 To: Dave Page Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: Re: [HACKERS] Win32 Powerfail testing - results Dave Page [EMAIL PROTECTED] writes: Rod Taylor allegedly said: Any

Re: [HACKERS] Win32 Powerfail testing - results

2003-02-03 Thread Hannu Krosing
Dave Page kirjutas E, 03.02.2003 kell 18:51: Well the results are finally in. Hopefully we can concentrate on putting them right, rather than having a round of told you so's :-) I modified the test program slightly to improve the consistency checks. The updated version is attached.

Re: [HACKERS] Win32 Powerfail testing - results

2003-02-03 Thread Dave Page
-Original Message- From: Hannu Krosing [mailto:[EMAIL PROTECTED]] Sent: 03 February 2003 22:30 To: Dave Page Cc: PostgreSQL Hackers; Katie Ward Subject: Re: [HACKERS] Win32 Powerfail testing - results Your hardware should also be able to run Postgres on BeOS http

Re: [HACKERS] Win32 Powerfail testing - results

2003-02-03 Thread Vince Vielhaber
On Mon, 3 Feb 2003, Dave Page wrote: Well the results are finally in. Hopefully we can concentrate on putting them right, rather than having a round of told you so's :-) I modified the test program slightly to improve the consistency checks. The updated version is attached. [...] Run |

Re: [HACKERS] Win32 Powerfail testing - results

2003-02-03 Thread Rod Taylor
I modified the test program slightly to improve the consistency checks. The updated version is attached. For curiosity sake, I've compiled it and am running it on FreeBSD with soft-updates enabled. A few variable declarations needed to be bumped up to the top of their respective function. Any

Re: [HACKERS] Win32 Powerfail testing - results

2003-02-03 Thread Dave Page
Rod Taylor allegedly said: I modified the test program slightly to improve the consistency checks. The updated version is attached. For curiosity sake, I've compiled it and am running it on FreeBSD with soft-updates enabled. A few variable declarations needed to be bumped up to the top of

Re: [HACKERS] Win32 Powerfail testing - results

2003-02-03 Thread Tom Lane
Dave Page [EMAIL PROTECTED] writes: Rod Taylor allegedly said: Any change of tossing in a periodic VACUUM or would that throw off the results? Dunno, Tom could best answer that, but a *complete guess* based on piecing together tidbits of how it all works from various threads here, would be