From the other server:
amandabackup@eclogite:~/daily$ cat
/usr/local/var/amanda/chg-robot-dev-tape-by-id-scsi-1BDT-FlexStor-II-00MX64100589-LL0
$STATE = {
'drive_lru' => [
'0',
'1'
],
'last_status' => 1602694135,
'last_operation_delay' => 0,
'last_operation_time' => 1602694089,
'slots' => {
'3' => {
'loaded_in' => undef,
'label' => 'geo-daily-062',
'state' => 1,
'ie' => 0,
'f_type' => bless( {
'sign' => '+',
'value' => [
1
]
}, 'Math::BigInt' ),
'device_error' => undef,
'barcode' => '000062L7',
'device_status' => bless( {
'value' => [
0
],
'sign' => '+'
}, 'Math::BigInt' )
},
'4' => {
'device_status' => bless( {
'sign' => '+',
'value' => [
0
]
}, 'Math::BigInt' ),
'barcode' => '000063L7',
'device_error' => undef,
'f_type' => bless( {
'value' => [
1
],
'sign' => '+'
}, 'Math::BigInt' ),
'ie' => 0,
'state' => 1,
'loaded_in' => undef,
'label' => 'geo-daily-063'
},
'17' => {
'device_status' => bless( {
'value' => [
4
],
'sign' => '+'
}, 'Math::BigInt' ),
'barcode' => '000076L7',
'device_error' => 'Tape device /dev/nst1
is not ready or is
empty',
'ie' => 0,
'f_type' => undef,
'state' => 1,
'loaded_in' => undef,
'label' => undef
},
'16' => {
'barcode' => '000075L7',
'device_status' => bless( {
'sign' => '+',
'value' => [
4
]
}, 'Math::BigInt' ),
'device_error' => 'Tape device /dev/nst1
is not ready or is
empty',
'ie' => 0,
'f_type' => undef,
'state' => 1,
'loaded_in' => undef,
'label' => undef
},
'10' => {
'ie' => 0,
'f_type' => undef,
'state' => 1,
'loaded_in' => undef,
'label' => undef,
'barcode' => '000069L7',
'device_status' => bless( {
'sign' => '+',
'value' => [
4
]
}, 'Math::BigInt' ),
'device_error' => 'Tape device /dev/nst1
is not ready or is
empty'
},
'13' => {
'device_error' => 'Tape device /dev/nst1
is not ready or is
empty',
'device_status' => bless( {
'value' => [
4
],
'sign' => '+'
}, 'Math::BigInt' ),
'barcode' => '000072L7',
'loaded_in' => undef,
'label' => undef,
'f_type' => undef,
'ie' => 0,
'state' => 1
},
'7' => {
'state' => 1,
'ie' => 0,
'f_type' => undef,
'loaded_in' => undef,
'label' => undef,
'barcode' => '000066L7',
'device_status' => bless( {
'value' => [
4
],
'sign' => '+'
}, 'Math::BigInt' ),
'device_error' => 'Tape device /dev/nst1
is not ready or is empty'
},
'8' => {
'device_error' => 'Tape device /dev/nst1
is not ready or is
empty',
'device_status' => bless( {
'sign' => '+',
'value' => [
4
]
}, 'Math::BigInt' ),
'barcode' => '000067L7',
'label' => undef,
'loaded_in' => undef,
'ie' => 0,
'f_type' => undef,
'state' => 1
},
'19' => {
'label' => undef,
'loaded_in' => undef,
'f_type' => undef,
'ie' => 0,
'state' => 1,
'device_error' => 'Tape device /dev/nst1
is not ready or is
empty',
'barcode' => '000078L7',
'device_status' => bless( {
'value' => [
4
],
'sign' => '+'
}, 'Math::BigInt' )
},
'23' => {
'ie' => 0,
'state' => 1,
'f_type' => undef,
'loaded_in' => undef,
'label' => undef,
'barcode' => '000050L7',
'device_status' => bless( {
'value' => [
4
],
'sign' => '+'
}, 'Math::BigInt' ),
'device_error' => 'Tape device /dev/nst1
is not ready or is
empty'
},
'6' => {
'device_error' => undef,
'label' => 'geo-daily-065',
'loaded_in' => '1',
'device_status' => bless( {
'value' => [
0
],
'sign' => '+'
}, 'Math::BigInt' ),
'barcode' => '000065L7',
'state' => 1,
'f_type' => bless( {
'sign' => '+',
'value' => [
1
]
}, 'Math::BigInt' )
},
'2' => {
'f_type' => bless( {
'value' => [
1
],
'sign' => '+'
}, 'Math::BigInt' ),
'ie' => 0,
'state' => 1,
'loaded_in' => undef,
'label' => 'geo-daily-061',
'barcode' => '000061L7',
'device_status' => bless( {
'sign' => '+',
'value' => [
0
]
}, 'Math::BigInt' ),
'device_error' => undef
},
'12' => {
'barcode' => '000071L7',
'device_status' => bless( {
'value' => [
4
],
'sign' => '+'
}, 'Math::BigInt' ),
'device_error' => 'Tape device /dev/nst1
is not ready or is
empty',
'f_type' => undef,
'ie' => 0,
'state' => 1,
'loaded_in' => undef,
'label' => undef
},
'14' => {
'label' => undef,
'loaded_in' => undef,
'ie' => 0,
'state' => 1,
'f_type' => undef,
'device_error' => 'Tape device /dev/nst1
is not ready or is
empty',
'barcode' => '000073L7',
'device_status' => bless( {
'sign' => '+',
'value' => [
4
]
}, 'Math::BigInt' )
},
'22' => {
'device_status' => bless( {
'sign' => '+',
'value' => [
4
]
}, 'Math::BigInt' ),
'barcode' => '000049L7',
'device_error' => 'Tape device /dev/nst1
is not ready or is
empty',
'ie' => 0,
'state' => 1,
'f_type' => undef,
'loaded_in' => undef,
'label' => undef
},
'20' => {
'device_status' => bless( {
'value' => [
4
],
'sign' => '+'
}, 'Math::BigInt' ),
'barcode' => '000079L7',
'device_error' => 'Tape device /dev/nst1
is not ready or is
empty',
'ie' => 0,
'state' => 1,
'f_type' => undef,
'label' => undef,
'loaded_in' => undef
},
'21' => {
'ie' => 0,
'state' => 1,
'f_type' => undef,
'loaded_in' => undef,
'label' => undef,
'device_status' => bless( {
'value' => [
4
],
'sign' => '+'
}, 'Math::BigInt' ),
'barcode' => '000048L7',
'device_error' => 'Tape device /dev/nst1
is not ready or is
empty'
},
'15' => {
'device_error' => 'Tape device /dev/nst1
is not ready or is
empty',
'device_status' => bless( {
'sign' => '+',
'value' => [
4
]
}, 'Math::BigInt' ),
'barcode' => '000074L7',
'loaded_in' => undef,
'label' => undef,
'ie' => 0,
'f_type' => undef,
'state' => 1
},
'1' => {
'device_status' => undef,
'barcode' => undef,
'device_error' => undef,
'ie' => 0,
'state' => 2,
'f_type' => undef,
'label' => undef,
'loaded_in' => undef
},
'11' => {
'barcode' => '000070L7',
'device_status' => bless( {
'value' => [
4
],
'sign' => '+'
}, 'Math::BigInt' ),
'device_error' => 'Tape device /dev/nst1
is not ready or is
empty',
'f_type' => undef,
'ie' => 0,
'state' => 1,
'label' => undef,
'loaded_in' => undef
},
'18' => {
'label' => undef,
'loaded_in' => undef,
'ie' => 0,
'state' => 1,
'f_type' => undef,
'device_error' => 'Tape device /dev/nst1
is not ready or is
empty',
'barcode' => '000077L7',
'device_status' => bless( {
'value' => [
4
],
'sign' => '+'
}, 'Math::BigInt' )
},
'5' => {
'device_error' => undef,
'barcode' => '000064L7',
'device_status' => bless( {
'value' => [
0
],
'sign' => '+'
}, 'Math::BigInt' ),
'label' => 'geo-daily-064',
'loaded_in' => undef,
'ie' => 0,
'state' => 1,
'f_type' => bless( {
'value' => [
1
],
'sign' => '+'
}, 'Math::BigInt' )
},
'24' => {
'device_error' => undef,
'device_status' => undef,
'barcode' => 'CLN031CU',
'loaded_in' => undef,
'label' => undef,
'ie' => 0,
'f_type' => undef,
'state' => 1
},
'9' => {
'device_error' => 'Tape device /dev/nst1
is not ready or is
empty',
'barcode' => '000068L7',
'device_status' => bless( {
'sign' => '+',
'value' => [
4
]
}, 'Math::BigInt' ),
'label' => undef,
'loaded_in' => undef,
'ie' => 0,
'state' => 1,
'f_type' => undef
}
},
'drives' => {
'1' => {
'label' => 'geo-daily-065',
'res_info' => undef,
'state' => 1,
'barcode' => '000065L7',
'orig_slot' => 6
},
'0' => {
'barcode' => undef,
'state' => 2,
'orig_slot' => undef,
'label' => undef
}
},
'bc2lb' => {
'000100L7' => 'geo-daily-100',
'000091L7' => 'geo-daily-091',
'000063L7' => 'geo-daily-063',
'000090L7' => 'geo-daily-090',
'000086L7' => 'geo-daily-086',
'000061L7' => 'geo-daily-061',
'000083L6' => 'geo-daily-223',
'000064L7' => 'geo-daily-064',
'000086L6' => 'geo-daily-226',
'000065L7' => 'geo-daily-065',
'000087L6' => 'geo-daily-227',
'000082L7' => 'geo-daily-082',
'000061L6' => 'geo-daily-201',
'000066L6' => 'geo-daily-206',
'000062L6' => 'geo-daily-202',
'000064L6' => 'geo-daily-204',
'000091L6' => 'geo-daily-231',
'000095L7' => 'geo-daily-095',
'000088L7' => 'geo-daily-088',
'000080L7' => 'geo-daily-080',
'000092L7' => 'geo-daily-092',
'000084L7' => 'geo-daily-084',
'000089L7' => 'geo-daily-089',
'000082L6' => 'geo-daily-222',
'000098L7' => 'geo-daily-098',
'000099L7' => 'geo-daily-099',
'000080L6' => 'geo-daily-220',
'000092L6' => 'geo-daily-232',
'000084L6' => 'geo-daily-224',
'000077L6' => 'geo-daily-217',
'000097L7' => 'geo-daily-097',
'000081L7' => 'geo-daily-081',
'000062L7' => 'geo-daily-062',
'000063L6' => 'geo-daily-203',
'000085L6' => 'geo-daily-225',
'000083L7' => 'geo-daily-083',
'000088L6' => 'geo-daily-228',
'000089L6' => 'geo-daily-229',
'000078L6' => 'geo-daily-218',
'000065L6' => 'geo-daily-205',
'000085L7' => 'geo-daily-085',
'000093L7' => 'geo-daily-093',
'000081L6' => 'geo-daily-221',
'000079L6' => 'geo-daily-219',
'000094L7' => 'geo-daily-094',
'000096L7' => 'geo-daily-096',
'000090L6' => 'geo-daily-230',
'000087L7' => 'geo-daily-087'
},
'current_slot' => {
'config' => {
'daily' => {
'storage' => {
'daily' => {
'changer' => {
'NEO200' => 6
}
}
}
}
}
}
};On 10/14/20 1:32 PM, Nathan Stratton Treadway wrote:
On Wed, Oct 14, 2020 at 12:20:33 -0400, Chris Hoogendyk wrote:Both lexical.pm and ScanInventory.pm restored to original. New fix only patch applied to ScanInventory.pm. amanda.conf restored to use oldest. amanda@marlin:~/daily$ amcheck daily Amanda Tape Server Host Check ----------------------------- NOTE: Holding disk '/amanda3': 139784192 KB disk space available, using 34926592 KB NOTE: Holding disk '/amanda4': 170082304 KB disk space available, using 65224704 KB NOTE: Holding disk '/amanda5': 240713728 KB disk space available, using 135856128 KB ***** Authorized Use Only ***** snapper slot 25: volume 'Bio-Research-028' Will write to volume 'Bio-Research-028' in slot 25. NOTE: skipping tape-writable test Server check took 50.059 seconds Amanda Backup Client Hosts Check -------------------------------- Client check: 4 hosts checked in 6.692 seconds. 0 problems found. (brought to you by Amanda 3.5.1) amanda@marlin:~/daily$[...]>Launched a flush on that. Then the following seems to set up a tape on the second tape drive. amanda@marlin:~/daily$ amtape daily taper slot 31: volume 'Bio-Research-032' Will write to volume 'Bio-Research-032' in slot 31. amanda@marlin:~/daily$Okay, sounds like things are back to working "normally" on that server, right? So, do you still have a second server which is getting coredumps (at least with the oldest taperscan)? Based on the investigation so far, it seems like the crash is caused by tape-inventory records which have no label text along with some specific other data field values. If you post the /usr/local/var/amanda/chg-robot-dev-tape-by-id-scsi* changer state file that other server, we can double check that such entries exist over there, too. (Assume they do, then I guess the question will be whether you want to apply the same ScanInventory.pm patch there, or if you instead want to try clearing that/those bad inventory record(s) without changing the installed code on that box....) Nathan ---------------------------------------------------------------------------- Nathan Stratton Treadway - [email protected] - Mid-Atlantic region Ray Ontko & Co. - Software consulting services - http://www.ontko.com/ GPG Key: http://www.ontko.com/~nathanst/gpg_key.txt ID: 1023D/ECFB6239 Key fingerprint = 6AD8 485E 20B9 5C71 231C 0C32 15F3 ADCD ECFB 6239
-- --------------- Chris Hoogendyk - O__ ---- Systems Administrator, Retired c/ /'_ --- Biology & Geosciences Departments (*) \(*) -- 315 Morrill Science Center III ~~~~~~~~~~ - University of Massachusetts, Amherst <[email protected]> --------------- Erdös 4
