On May 14, 2008, at 11:57 PM, Greg Connor wrote:
[Saving large files with mogtool]
Here's the error generated by mogtool.
WARNING: Unable to save file 'collect-20080513-vol9,2201': Unable to
create new file at /usr/bin/mogtool line 812, <Sock_minime336:7001>
line 11907.
MogileFS backend error message: unknown_key unknown_key
System error message: Unable to create new file at /usr/bin/mogtool
line 812, <Sock_minime336:7001> line 11907.
This was try #9707 and it's been 10578.37 seconds since we first
tried. Retrying...
What does the backend logs say?
There's a bug in the latest released mogtool that makes the automatic
retrying on failure not work. Patch below; the change is also in SVN
(as per r1130).
I forget the details, but I noticed the bug when my backends had some
sort of configuration problem that made them fail once in a while.
- ask
--- mogtool (revision 1129)
+++ mogtool (revision 1130)
@@ -799,25 +799,26 @@
my $mogfs = get_mogfs();
my $dkey = $opts{big} ? "$key,$chunknum" : "$key";
- # TODO: be resilient to transient errors, retry, etc.
my $start_time = [ gettimeofday() ];
my $try = 0;
while (1) {
$try++;
- my $fh = $mogfs->new_file($dkey, $opts{class}, $bufsize);
- unless (defined $fh) {
- error("WARNING: Unable to create new file '$dkey'.");
+ eval {
+ my $fh = $mogfs->new_file($dkey, $opts{class},
$bufsize);
+ unless (defined $fh) {
+ die "Unable to create new file";
+ }
+ $fh->print($chunkbuf{$cn});
+ unless ($fh->close) {
+ die "Close failed";
+ }
+ };
+ if (my $err = $@) {
+ error("WARNING: Unable to save file '$dkey': $err");
printf "This was try #$try and it's been %.2f
seconds since we first tried. Retrying...\n", tv_interval($start_time);
sleep 1;
next;
}
- $fh->print($chunkbuf{$cn});
- unless ($fh->close) {
- error("WARNING: Unable to save file '$dkey'.");
- printf "This was try #$try and it's been %.2f seconds
since we first tried. Retrying...\n", tv_interval($start_time);
- sleep 1;
- next;
- }
last;
}
my $diff = tv_interval($start_time);
--
http://develooper.com/ - http://askask.com/
