On Monday 17 November 2003 11:16, Bruce Momjian wrote: > Tom Lane wrote: > > > Do we know that having the background writer fsync a file that was > > > written by a backend cause all the data to fsync? I think I could > > > write a program to test this by timing each of these tests: > > > > That might prove something about the particular platform you tested it > > on; but it would not speak to the real problem, which is what we can > > assume is true on every platform... > > The attached program does test if fsync can be used on a file descriptor > after the file is closed and then reopened. I see: > > write 0.000613 > write & fsync 0.001727 > write, close & fsync 0.001633
ArchLinux, maxtor IDE HDD, write cache enabled. [EMAIL PROTECTED] tmp]$ gcc -o test_fsync test_fsync.c [EMAIL PROTECTED] tmp]$ ./test_fsync write 0.002403 write & fsync 0.009423 write, close & fsync 0.006457 [EMAIL PROTECTED] tmp]$ uname -a Linux daithan 2.4.21 #1 SMP Tue Jul 8 19:41:52 PDT 2003 i686 unknown > Anyway, if we find all our platforms can pass this test, we might be > able to allow backends to do their own writes and just record the file > name somewhere for the checkpointer to fsync. It also shows write/fsync > was 3x slower than simple write. > > Does anyone have a platform where the last duration is significantly > different from the middle timing? Does 30% difference above count as significant? Assuming fsync on a file descriptor flushes dirty buffers of that file, from all processes, would following be sufficient? 1. Open WAL with O_SYNC|O_DIRECT (Later whereever possible) And issue fsync on WAL files whenever required. 2. Use regular writes for data files and fsync them in background. May be if background process is the only one that issues any fsync on data files, that could maximize overall system throughput. Say, all backends write to a datafile and signal the background writer, that they are blocked on this write to complete. BGWriter could chunk all such requests and flush them/fsync when there is enough disk activity. Hopefully none of them would be stalled for too long. That way slowest part of the system i.e the disk will be kept full of load. Besides since WAL writes are synchornous, backgrounds can safely push a write and move to further business, most of the times. I guess BGWriter has to fsync the data files anyways to recycle a WAL segment. In idle conditions, this mechanism should not be a problem. Just a thought. Does this take care of sync? > I am keeping this discussion on patches because of the C program > attachment. I dropped win32 list. I am not subscribed to it. Just getting thread out of it. I will write a short program which writes to a file in different processes and attempts to fsync them from only one. Let's see what that turns out. Shridhar ---------------------------(end of broadcast)--------------------------- TIP 8: explain analyze is your friend
