Re: [HACKERS] Win32 Powerfail testing

2003-03-07 Thread Greg Stark
Kevin Brown <[EMAIL PROTECTED]> writes: > Even that's not strictly necessary -- we *do* have shared memory we > can use for this, and even when hundreds of tables have been written > the list will only end up being a few tens of kilobytes in size (plus > whatever overhead is required to track and

Re: [HACKERS] Win32 Powerfail testing

2003-03-07 Thread Tatsuo Ishii
> But even then, we don't actually have to track the *names* of the > files that have changed, just their RelFileNodes, since there's a > mapping function from the RelFileNode to the filename. Right. I have noticed that too and have made changes to my implementaion. BTW, you need to track the blo

Re: [HACKERS] Win32 Powerfail testing

2003-03-07 Thread Kevin Brown
Bruce Momjian wrote: > Kevin Brown wrote: > > Bruce Momjian wrote: > > > The idea of using this on Unix is tempting, but Tatsuo is using a > > > threaded backend, so it is a little easier to do. However, it would > > > probably be pretty easy to write a file of modified file names that the > > > c

Re: [HACKERS] Win32 Powerfail testing

2003-03-07 Thread Bruce Momjian
Kevin Brown wrote: > Bruce Momjian wrote: > > The idea of using this on Unix is tempting, but Tatsuo is using a > > threaded backend, so it is a little easier to do. However, it would > > probably be pretty easy to write a file of modified file names that the > > checkpoint could read and open/fsy

Re: [HACKERS] Win32 Powerfail testing

2003-03-07 Thread Kevin Brown
Bruce Momjian wrote: > The idea of using this on Unix is tempting, but Tatsuo is using a > threaded backend, so it is a little easier to do. However, it would > probably be pretty easy to write a file of modified file names that the > checkpoint could read and open/fsync/close. Even that's not st

Re: [HACKERS] Win32 Powerfail testing

2003-03-07 Thread Bruce Momjian
Kevin Brown wrote: > Tatsuo Ishii wrote: > > Today I revisited the implemnetation (replacing sync() with > > open/_commit) I made several days ago and found a bug with it (thanks > > to Hiroshi). With the fixed version of it, now my Win32 port has > > passed your test even right after checkpoint!.

Re: [HACKERS] Win32 Powerfail testing

2003-03-07 Thread Tatsuo Ishii
> > I presume that this implementation tracks which files have been opened > > and uses _commit() to write all the changes to disk for those files? > > But are there quarantees that all closed files are flushed to disk as > well ? > > Does postgres quarantee it by doing a _commit() before close()

Re: [HACKERS] Win32 Powerfail testing

2003-03-07 Thread Hannu Krosing
Kevin Brown kirjutas R, 07.03.2003 kell 12:05: > Tatsuo Ishii wrote: > > Today I revisited the implemnetation (replacing sync() with > > open/_commit) I made several days ago and found a bug with it (thanks > > to Hiroshi). With the fixed version of it, now my Win32 port has > > passed your test ev

Re: [HACKERS] Win32 Powerfail testing

2003-03-07 Thread Kevin Brown
Tatsuo Ishii wrote: > Today I revisited the implemnetation (replacing sync() with > open/_commit) I made several days ago and found a bug with it (thanks > to Hiroshi). With the fixed version of it, now my Win32 port has > passed your test even right after checkpoint!. I presume that this implemen

Re: [HACKERS] Win32 Powerfail testing

2003-03-06 Thread Tatsuo Ishii
> > As I said in the previlus mails, open()+_commit() does the > > right job with the transaction log files. So probably I think > > I should stick with open()+_commit() approach for ordinary > > table/index files too. > > Oh, I didn't see that message. So it's either: > > open() + _commit()

Re: [HACKERS] Win32 Powerfail testing

2003-03-06 Thread Merlin Moncure
erlin > -Original Message- > From: Dave Page [mailto:[EMAIL PROTECTED] > Sent: Thursday, March 06, 2003 11:02 AM > To: Tatsuo Ishii > Cc: [EMAIL PROTECTED] > Subject: Re: [HACKERS] Win32 Powerfail testing > > > > > -Original Message- > > From: Tatsuo

Re: [HACKERS] Win32 Powerfail testing

2003-03-06 Thread Tatsuo Ishii
> > Are you asking the way how to open files in the buffer > > manager? If so, basically PostgreSQL uses open() with flags > > (O_RDWR | PG_BINARY, 0600). > > I cannot find it now, but I'm sure I read that FlushFileBuffers() has no > effect unless the file was opened with CreateFile() with the >

Re: [HACKERS] Win32 Powerfail testing

2003-03-06 Thread Magnus Hagander
> Agreed, but I still keep thinking that despite some peoples > claims that Windows ain't up to it, DB2, SQL and Exchange > Server as well a probably others that don't use raw > partitions have got over this problem, so therefore we should > be able to. Admittedly Microsoft have a bit of an adv

Re: [HACKERS] Win32 Powerfail testing

2003-03-06 Thread Dave Page
> -Original Message- > From: Tatsuo Ishii [mailto:[EMAIL PROTECTED] > Sent: 06 March 2003 14:00 > To: Dave Page > Cc: [EMAIL PROTECTED] > Subject: Re: [HACKERS] Win32 Powerfail testing > > > > > Sorry, but it does not help. The page says we could use

Re: [HACKERS] Win32 Powerfail testing

2003-03-06 Thread Tatsuo Ishii
> > Sorry, but it does not help. The page says we could use > > FlushFileBuffers() to sync the kernel buffer to the > > disk. Unfortunately, it requires a file descriptor to flush > > for its argument. Thus it could not be a replacement of > > sync(). Actually I have modified the buffer manager s

Re: [HACKERS] Win32 Powerfail testing

2003-03-06 Thread Dave Page
> -Original Message- > From: Tatsuo Ishii [mailto:[EMAIL PROTECTED] > Sent: 05 March 2003 13:49 > To: Dave Page > Cc: [EMAIL PROTECTED] > Subject: Re: [HACKERS] Win32 Powerfail testing > > > > > So far we found interesting facts. Our Win32 port pa

Re: [HACKERS] Win32 Powerfail testing

2003-03-06 Thread Dave Page
> -Original Message- > From: Kevin Brown [mailto:[EMAIL PROTECTED] > Sent: 06 March 2003 04:37 > To: [EMAIL PROTECTED] > Subject: Re: [HACKERS] Win32 Powerfail testing > > > Tatsuo Ishii wrote: > > Sorry, but it does not help. The page says we could use

Re: [HACKERS] Win32 Powerfail testing

2003-03-06 Thread Tatsuo Ishii
> It would be an interesting comparison for you to roll the file > descriptor tracking changes into the Unix side of the tree and use > fsync() or fdatasync() in place of FlushFileBuffers() on the Unix side > (you'd have to remove or disable the code that does a sync() of > course). If the end res

Re: [HACKERS] Win32 Powerfail testing

2003-03-05 Thread Kevin Brown
Tatsuo Ishii wrote: > Sorry, but it does not help. The page says we could use > FlushFileBuffers() to sync the kernel buffer to the > disk. Unfortunately, it requires a file descriptor to flush for its > argument. Thus it could not be a replacement of sync(). Actually I > have modified the buffer m

Re: [HACKERS] Win32 Powerfail testing

2003-03-05 Thread scott.marlowe
On Wed, 5 Mar 2003, Dave Page wrote: > > > > -Original Message- > > From: Tatsuo Ishii [mailto:[EMAIL PROTECTED] > > Sent: 05 March 2003 02:23 > > To: [EMAIL PROTECTED] > > Subject: [HACKERS] Win32 Powerfail testing > > > > So far we fo

[HACKERS] Win32 Powerfail testing

2003-03-05 Thread Tatsuo Ishii
We are developing a Win32 port of PostgreSQL 7.3(different from Jan's implementaion, in that we are using a thread model. In the future I hope we could contribute the source code). We have done a power failure testing using the test tool made by Dave Page: Subject: [HACKERS] Win32 Powe

Re: [HACKERS] Win32 Powerfail testing - results

2003-02-03 Thread Dave Page
> -Original Message- > From: Hannu Krosing [mailto:[EMAIL PROTECTED]] > Sent: 03 February 2003 22:30 > To: Dave Page > Cc: PostgreSQL Hackers; Katie Ward > Subject: Re: [HACKERS] Win32 Powerfail testing - results > > > Your hardware should also be a

Re: [HACKERS] Win32 Powerfail testing - results

2003-02-03 Thread Hannu Krosing
Dave Page kirjutas E, 03.02.2003 kell 18:51: > Well the results are finally in. Hopefully we can concentrate on putting > them right, rather than having a round of "told you so's" :-) > > I modified the test program slightly to improve the consistency checks. > The updated version is attached. >

Re: [HACKERS] Win32 Powerfail testing - results

2003-02-03 Thread Dave Page
> -Original Message- > From: Tom Lane [mailto:[EMAIL PROTECTED]] > Sent: 03 February 2003 21:52 > To: Dave Page > Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED] > Subject: Re: [HACKERS] Win32 Powerfail testing - results > > > "Dave Page" <[E

Re: [HACKERS] Win32 Powerfail testing - results

2003-02-03 Thread Tom Lane
"Dave Page" <[EMAIL PROTECTED]> writes: > Rod Taylor allegedly said: >> Any change of tossing in a periodic VACUUM or would that throw off the >> results? > Dunno, Tom could best answer that, but a *complete guess* based on piecing > together tidbits of how it all works from various threads here,

Re: [HACKERS] Win32 Powerfail testing - results

2003-02-03 Thread Dave Page
Rod Taylor allegedly said: >> I modified the test program slightly to improve the consistency >> checks. The updated version is attached. > > For curiosity sake, I've compiled it and am running it on FreeBSD with > soft-updates enabled. > > A few variable declarations needed to be bumped up to the

Re: [HACKERS] Win32 Powerfail testing - results

2003-02-03 Thread Dave Page
Vince Vielhaber allegedly said: > On Mon, 3 Feb 2003, Dave Page wrote: > >> Run | Errors Detected >> = >> 07 | COUNT CHECK - Duplicate or missing rows detected (10262)!! 09 | >> DISTINCT CHECK - Duplicate or missing rows detected (9893)

Re: [HACKERS] Win32 Powerfail testing - results

2003-02-03 Thread Rod Taylor
> I modified the test program slightly to improve the consistency checks. > The updated version is attached. For curiosity sake, I've compiled it and am running it on FreeBSD with soft-updates enabled. A few variable declarations needed to be bumped up to the top of their respective function. An

Re: [HACKERS] Win32 Powerfail testing - results

2003-02-03 Thread Vince Vielhaber
On Mon, 3 Feb 2003, Dave Page wrote: > Well the results are finally in. Hopefully we can concentrate on putting > them right, rather than having a round of "told you so's" :-) > > I modified the test program slightly to improve the consistency checks. > The updated version is attached. [...] > >

[HACKERS] Win32 Powerfail testing - results

2003-02-03 Thread Dave Page
Well the results are finally in. Hopefully we can concentrate on putting them right, rather than having a round of "told you so's" :-) I modified the test program slightly to improve the consistency checks. The updated version is attached. Regards, Dave. System == Gigabyte GA-6VTXD Motherbo