This is in response to the -autocdb problems reported by Brian Lalor on tmda-users.
Mike's analysis looks pretty good to me. Any problems before I work on a fix? BTW, I think -autodbm should be OK since I'm already using tempfile.mktemp() in Util.build_dbm(). Not sure why I didn't do this for Util.build_cdb() as well.. -------------------- Start of forwarded message -------------------- Date: Sun, 10 Nov 2002 02:41:38 -0600 (CST) From: "M.J. Pomraning" <[EMAIL PROTECTED]> To: "Jason R. Mastaler" <[EMAIL PROTECTED]> Subject: tmda-filter -autocdb race Short: cdb = cdb.cdbmake(cdbname, cdbname + ".tmp." + str(os.getpid())) Long: Two concurrent tmda-filter processes remaking the same -autocdb file, possibly in response to a single CONFIRM_APPEND by a third process, call cdb.cdbmake with the same second argument. This arg sets cdbmake_obj.fntmp, the scratch filename for cdb generation. Whichever finishes first atomically rename()s fntmp to the target filename. This file is incomplete, however, having been whomped and only partially written by the second process. Any cdb reader opening the target file while the second process is still working might well try to look past the end of an under-construction file: EPROTO, in this case. The second process promptly finishes up, fails to rename() the (now non-existent) fntmp, and throws an exception causing build_cdb() to return false. This is recovered from in __autobuild_db() without complaint (default to text file search), and future cdb file readers are none-the-wiser. What do you think? I didn't look too closely (and it's late), but I didn't see any interprocess logic to prevent this. Barring threads/NFS/that kind of thing, a little per-process salt should sufficiently uniquify fntmp, or a little tempfile.mktemp() for that matter. Come to think of it, the naive ".tmp" is probably taken from the python-cdb Example file. Drat. Regards, Mike -- MJ Pomraning [EMAIL PROTECTED] http://pilcrow.madison.wi.us -------------------- End of forwarded message -------------------- _________________________________________________ tmda-workers mailing list ([EMAIL PROTECTED]) http://tmda.net/lists/listinfo/tmda-workers
