Hi!
On Fri, 2010-02-26 at 21:55:41 +0100, Jean-Baptiste Lallement wrote:
> I tested the fsync patch. There is one issue in processarc.c. The
> call to dir_sync_contents fails because at this point 'cidir' is the
> full path of the control file, not the name of the directory
> containing the control file.
Nice catch!
> Following your work, I added a dir_sync_contents_parent in
> lib/dpkg/dir.{c,h} and replaced the call to dir_sync_contents in
> process_archive. I'm not sure if it's done the way you're expecting,
> but it builds and run fine. The patch is attached.
Actually there was an easier fix for this, just moving the
dir_sync_contents call before the “strcpy(cidirrest,CONTROLFILE);”
call, whichs is appending the control file name to the directory name.
> I tested the following cases: install, upgrade, removal and purge.
> Then ran a sha256sum on the files (archive, control files, and dpkg
> db) and compared to the expected checksum. All the files are
> correctly synced or removed without residual files left behind (some
> .dpkg-new or status-new)
Ah, perftect.
> On 02/17/2010 09:09 PM, Guillem Jover wrote:
> > doing too many fsync()s on directories, and I'll probably switch that
> > to mark them as dirty and only fsync() them then.
> I confirm that there are too many syncs on directories. Some
> directories are sync as many times as the number of files it
> contains. Marking them as dirty when the status of the package is
> set 'unpacked' should fix it. You'll find the number of sync/file
> for bsdgames at http://www.pastebin.Com/NdHNWsyQ
Yeah, after reviewing the patch one more time, I've seen that too.
I've committed the database dir sync part for now, and have a
preliminary patch for the dir sync on all file system dirty
directories. But that one can still wait a bit.
> > I think I'll split it in few pieces, one for the missing fsync()/fflush()
> > on files, another one for dir fsync()s on db paths, and a last one to
> > be worked on for fsync()s on the installed directories. This way we get
> > incremental improvements, also I've not yet checked the impact of the
> > additional fsync()s, which might be notibable for the installed files.
> The major issue seems to be the performance. I tested
> texlive-latex-extra-doc (more than 8000 files) some directories are
> sync around 500 times. I've only tested on a VM up to now, but the
> performance loss is a factor 3. Take this number with caution, it's
> a VM with a virtual disk, not a raw device hence disk access does
> not means much. I'll do some profiling on a real machine this week
> end.
Right, I think we'd need to test the impact of the fsync on all dirty
directories on the file system before applying it. It might be too
much.
And thanks so much for all the testing!
thanks,
guillem
--
To UNSUBSCRIBE, email to [email protected]
with a subject of "unsubscribe". Trouble? Contact [email protected]
