Hi Jim: And even if it is (releasing the lock, all orderly and everything), I'd generally never use anything but a zero-length shadow file for this kind of locking action, to elaborate on Kenneth's suggestion. If you're building up the archive, and if it's getting sizeable, it's possible that flock or one of its preambles is doing a stat on the file that is requiring a check of the block structure of the file, swinging from volume extent to volume extent, etc. Under ordinary circumstances that might be fine, but in your 80 ms. window that might be experiencing disk seek latency, etc., that is interacting with whatever in god's name tar is doing with the file or whatever state or fragmentation it's being left in....end result is you'd at best provide some latency to the locking atomicity,and usually increase the chance of finding windows where it would get into race conditions.
In fact, if the file stays zero length, your file system driver should pretty much keep it as a descriptor in memory for most of your running including any prelock stat(), which I'd think at your level of multithreading I/O activity, couldn't help but be a win. Regards, Wayne From: Kenneth Graves <[email protected]> To: [email protected], Cc: Boston Perl Mongers <[email protected]> Date: 04/06/2012 01:48 PM Subject: Re: [Boston.pm] Perl flock problem? Sent by: [email protected] Is tar releasing the lock? You might try using flock on a separate lock file instead of locking the archive itself. --kag On Apr 6, 2012, at 5:56 AM, [email protected] wrote: > we're running a pre-forking server with a number of child > processes which, among other things, archive incoming files > to a common archive file. Contention for the tar file is > handled by perl's flock: > > .... > my $fh = IO::File->new(">> $archdir/$arch_filename") > || die "Can't open archive file: > $archdir/$arch_filename : $!\n"; > flock($fh, LOCK_EX); > system('tar', $opts, "$archdir/$arch_filename", > "$basefilename") == 0 > || die "Can't add $basefilename to > $archdir/$arch_filename - $!"; > flock($fh, LOCK_UN); > $fh->close(); > .... > > We are finding from the log that when competing child > processes execute this code within the same 80 msec. or less > window that a directory checksum within the tar gets > corrupted unrecoverably since tar spits out this error > before exiting: > > directory checksum error > > Once this happens the archive is unusable. Outside of that > contention time band there's *never* a directory checksum > error. > > I can't see any problem with the perl or the use of flock > and yet the circumstances seem to suggest some sort of > problem with the file locking. Am I missing something? > This runs on HPUX 11.11 and invokes HPUX's tar. Is there any > known problem with perl's flock on this system or the > interaction of flock and tar? > > TIA, > > Jim Eshelman > > > > > > > > > _______________________________________________ > Boston-pm mailing list > [email protected] > http://mail.pm.org/mailman/listinfo/boston-pm _______________________________________________ Boston-pm mailing list [email protected] http://mail.pm.org/mailman/listinfo/boston-pm _______________________________________________ Boston-pm mailing list [email protected] http://mail.pm.org/mailman/listinfo/boston-pm
