Thanks Wayne, I think that's good advice. ----- Original Message Follows ----- From: Wayne Tackabury <[email protected]> To: Kenneth Graves <[email protected]> Cc: [email protected], [email protected], Boston Perl Mongers <[email protected]> Subject: Re: [Boston.pm] Perl flock problem? Date: Fri, 6 Apr 2012 17:26:43 -0400
> Hi Jim: > > And even if it is (releasing the lock, all orderly and > everything), I'd generally never use anything but a > zero-length shadow file for this kind of locking action, > to elaborate on Kenneth's suggestion. If you're building > up the archive, and if it's getting sizeable, it's > possible that flock or one of its preambles is doing a > stat on the file that is requiring a check of the block > structure of the file, swinging from volume extent to > volume extent, etc. Under ordinary circumstances that > might be fine, but in your 80 ms. window that might be > experiencing disk seek latency, etc., that is interacting > with whatever in god's name tar is doing with the file or > whatever state or fragmentation it's being left in....end > result is you'd at best provide some latency to the > locking atomicity,and usually increase the chance of > finding windows where it would get into race conditions. > > In fact, if the file stays zero length, your file system > driver should pretty much keep it as a descriptor in > memory for most of your running including any prelock > stat(), which I'd think at your level of multithreading > I/O activity, couldn't help but be a win. > > Regards, > Wayne > > > > From: Kenneth Graves <[email protected]> > To: [email protected], > Cc: Boston Perl Mongers <[email protected]> > Date: 04/06/2012 01:48 PM > Subject: Re: [Boston.pm] Perl flock problem? > Sent by: > [email protected] > > > > Is tar releasing the lock? You might try using flock on a > separate lock file instead of locking the archive itself. > > --kag > > On Apr 6, 2012, at 5:56 AM, [email protected] wrote: > > we're running a pre-forking server with a number of > > child processes which, among other things, archive > > incoming files to a common archive file. Contention > > for the tar file is handled by perl's flock: > > > > .... > > my $fh = IO::File->new(">> > > $archdir/$arch_filename") || die "Can't open > > archive file: $archdir/$arch_filename : $!\n"; > > flock($fh, LOCK_EX); > > system('tar', $opts, "$archdir/$arch_filename", > > "$basefilename") == 0 > > || die "Can't add $basefilename to > > $archdir/$arch_filename - $!"; > > flock($fh, LOCK_UN); > > $fh->close(); > > .... > > > > We are finding from the log that when competing child > > processes execute this code within the same 80 msec. or > > less window that a directory checksum within the tar > > gets corrupted unrecoverably since tar spits out this > > error before exiting: > > > > directory checksum error > > > > Once this happens the archive is unusable. Outside of > > that contention time band there's *never* a directory > > checksum error. > > > > I can't see any problem with the perl or the use of > > flock and yet the circumstances seem to suggest some > > sort of problem with the file locking. Am I missing > > something? This runs on HPUX 11.11 and invokes HPUX's > > tar. Is there any known problem with perl's flock on > > this system or the interaction of flock and tar? > > > > TIA, > > > > Jim Eshelman > > > > > > > > > > > > > > > > > > _______________________________________________ > > Boston-pm mailing list > > [email protected] > > http://mail.pm.org/mailman/listinfo/boston-pm > > > _______________________________________________ > Boston-pm mailing list > [email protected] > http://mail.pm.org/mailman/listinfo/boston-pm > > > > > _______________________________________________ > Boston-pm mailing list > [email protected] > http://mail.pm.org/mailman/listinfo/boston-pm _______________________________________________ Boston-pm mailing list [email protected] http://mail.pm.org/mailman/listinfo/boston-pm
