That's an interesting point too. In our case the archive name is date-based and a new archive file gets created in the first transaction after midnight. The contention failures we've experienced have all occurred later in the day, when the archive already exists, with many files already in it, so that particular problem is probably not affecting us.
----- Original Message Follows ----- From: Duane Bronson <[email protected]> To: [email protected] Cc: Boston Perl Mongers <[email protected]>, Kenneth Graves <[email protected]> Subject: Re: [Boston.pm] Perl flock problem? Date: Fri, 6 Apr 2012 15:55:35 -0400 > Possibly, you are locking AFTER the file is getting > created, so it's possible two processes will create the > file at about the same time (one will get unlinked from > the filesystem), and then both file handles will obtain a > different lock. > > The suggestion to lock a different file will have the same > problem if the file needs to be created first. Maybe you > want to make sure the file exists first with > > sysopen (LOCKFILE, "$archdir/$arch_filename", > O_RDWR|O_EXCL|O_CREAT, 0755) && close (LOCKFILE); # fails > if exists > > Duane > > On Apr 6, 2012, at 10:20 AM, [email protected] wrote: > > > Good point, could be. We'll try that. Thanks. > > > > > > ----- Original Message Follows ----- > > From: Kenneth Graves <[email protected]> > > To: [email protected] > > Cc: Boston Perl Mongers <[email protected]> > > Subject: Re: [Boston.pm] Perl flock problem? > > Date: Fri, 6 Apr 2012 12:47:56 -0500 > > > >> Is tar releasing the lock? You might try using flock > on a >> separate lock file instead of locking the archive > itself. >> > >> --kag > >> > >> On Apr 6, 2012, at 5:56 AM, [email protected] wrote: > >>> we're running a pre-forking server with a number of > >>> child processes which, among other things, archive > >>> incoming files to a common archive file. Contention > >>> for the tar file is handled by perl's flock: > >>> > >>> .... > >>> my $fh = IO::File->new(">> > >>> $archdir/$arch_filename") || die "Can't open > >>> archive file: $archdir/$arch_filename : $!\n"; > >>> flock($fh, LOCK_EX); > >>> system('tar', $opts, "$archdir/$arch_filename", > >>> "$basefilename") == 0 > >>> || die "Can't add $basefilename to > >>> $archdir/$arch_filename - $!"; > >>> flock($fh, LOCK_UN); > >>> $fh->close(); > >>> .... > >>> > >>> We are finding from the log that when competing child > >>> processes execute this code within the same 80 msec. > or >>> less window that a directory checksum within the > tar >>> gets corrupted unrecoverably since tar spits out > this >>> error before exiting: > >>> > >>> directory checksum error > >>> > >>> Once this happens the archive is unusable. Outside of > >>> that contention time band there's *never* a directory > >>> checksum error. > >>> > >>> I can't see any problem with the perl or the use of > >>> flock and yet the circumstances seem to suggest some > >>> sort of problem with the file locking. Am I missing > >>> something? This runs on HPUX 11.11 and invokes HPUX's > >>> tar. Is there any known problem with perl's flock on > >>> this system or the interaction of flock and tar? > >>> > >>> TIA, > >>> > >>> Jim Eshelman > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> _______________________________________________ > >>> Boston-pm mailing list > >>> [email protected] > >>> http://mail.pm.org/mailman/listinfo/boston-pm > >> > >> > >> _______________________________________________ > >> Boston-pm mailing list > >> [email protected] > >> http://mail.pm.org/mailman/listinfo/boston-pm > > > > _______________________________________________ > > Boston-pm mailing list > > [email protected] > > http://mail.pm.org/mailman/listinfo/boston-pm > > -- > > Duane Bronson > [email protected] > http://www.nerdlogic.com/ > 453 Washington St. #4A > Boston, MA 02111 > 617.515.2909 > > > _______________________________________________ > Boston-pm mailing list > [email protected] > http://mail.pm.org/mailman/listinfo/boston-pm _______________________________________________ Boston-pm mailing list [email protected] http://mail.pm.org/mailman/listinfo/boston-pm
