On 2017-11-14 14:37, Austin S. Hemmelgarn wrote:
On 2017-11-14 07:43, Austin S. Hemmelgarn wrote:
On 2017-11-14 07:34, Austin S. Hemmelgarn wrote:
On 2017-11-13 16:42, Jean-Louis Martineau wrote:
On 13/11/17 02:53 PM, Austin S. Hemmelgarn wrote:
driver: send-cmd time 9300.326 to taper1: VAULT-WRITE worker1-0
00-00120 local-vtl local-vtl Home-0001 client0 /home/1D 0
20171113073255 "" "" "" "" 1073741824 memory "" "" 0
> FAIL taper "ST:cloud" "POOL:cloud" client0 /home/1D
20171113073255 0 error "File 0 not found"
Do that dump still exists on tape Home-0001? Find it with amfetchdump.
If yes, send me the taper debug file.
amfetchdump does not see it, but looking directly at the virtual tape
directories, I can see it there.
Just tried an amcheckdump on everything, it looks like some of the
dump files are corrupted, but I can't for the life of me figure out
why (I test our network regularly and it has no problems, and any
problems with a particular system should show up as more than just
corrupted tar files). I'm going to try disabling compression and see
if that helps at all, as that's the only processing other than the
default that we're doing on the dumps (long term, it's not really a
viable option, but if it fixes things at least we know what's broken).
No luck changing compression. I would suspect some issue with NFS, but
I've started seeing the same symptoms on my laptop as well now (which is
completely unrelated to any of the sets at work other than having an
almost identical configuration other than paths and the total number of
tapes).
So, I finally got things working by switching from:
storage "local-vtl"
vault-storage "cloud"
To:
storage: "local-vtl" "cloud"
And removing the "vault" option from the local-vtl storage definition.
Strictly speaking, this is working around the issue instead of fixing
it, but it fits within what we need for our usage, and actually makes
the amdump runs complete faster (since dumps get taped to S3 in parallel
with getting taped to the local vtapes).
Based on this, and the fact that the issues I was seeing with corrupted
dumps being reported by amcheckdump, I think the issue is probably an
interaction between the vaulting code and the regular taping code, but
I'm not certain.
Thanks for the help.