The list of files with the 'slot-bb2e601e63b3408bb6a865e442a28366-6'
prefix (slot-6) show there is no data in the slot, only the tape label.
There is data only on slot-10,11,12 and 13.
I have no idea why there is no data in the others slots.
I assume the data was there after the backup, but I don't how they got
erased.
Jean-Louis
On 29/08/17 07:31 AM, Giorgio Valoti wrote:
> Il giorno 29 ago 2017, alle ore 12:36, Jean-Louis Martineau
<jmartin...@carbonite.com> ha scritto:
>
> This error often means that some s3 objects are missing.
>
> Do some of the data was restored?
> Can you post the amfetchdump debug file?
Here it is:
Tue Aug 29 08:27:19 2017: thd-0x1a8a600: amfetchdump: pid 26587 ruid
34 euid 34 version 3.3.6: start at Tue Aug 29 08:27:19 2017
Tue Aug 29 08:27:19 2017: thd-0x1a8a600: amfetchdump: Arguments: -p -a
zfs-dataset ci-master rpool/jenkins 20170822000017
Tue Aug 29 08:27:19 2017: thd-0x1a8a600: amfetchdump: pid 26587 ruid
34 euid 34 version 3.3.6: rename at Tue Aug 29 08:27:19 2017
Tue Aug 29 08:27:19 2017: thd-0x1a8a600: amfetchdump: Using state
file: /var/lib/amanda/state/s3-statefile
Tue Aug 29 08:27:19 2017: thd-0x1a8a600: amfetchdump:
Amanda::Recovery::Clerk: loading volume 'slot-6'
Tue Aug 29 08:27:19 2017: thd-0x1a8a600: amfetchdump: find_volume
labeled 'slot-6'
Tue Aug 29 08:27:19 2017: thd-0x1a8a600: amfetchdump: parse_inventory:
load slot 6 with label 'slot-6'
Tue Aug 29 08:27:19 2017: thd-0x1a8a600: amfetchdump: S3 driver using
bucket 'onebip-backups-admin', prefix
'slot-bb2e601e63b3408bb6a865e442a28366-6'
Tue Aug 29 08:27:19 2017: thd-0x1a8a600: amfetchdump: curl version:
libcurl/7.47.0 OpenSSL/1.0.2g zlib/1.2.8 libidn/1.32 librtmp/2.3
Tue Aug 29 08:27:19 2017: thd-0x1a8a600: amfetchdump: curl compiled
for OPENSSL
Tue Aug 29 08:27:19 2017: thd-0x1a8a600: amfetchdump: Create 8 threads
Tue Aug 29 08:27:19 2017: thd-0x1a8a600: amfetchdump: GET
https://onebip-backups-admin.s3.amazonaws.com/slot-bb2e601e63b3408bb6a865e442a28366-6f00000003-filestart
<https://onebip-backups-admin.s3.amazonaws.com/slot-bb2e601e63b3408bb6a865e442a28366-6f00000003-filestart>
failed with 404/NoSuchKey
Tue Aug 29 08:27:19 2017: thd-0x1a8a600: amfetchdump: GET
https://onebip-backups-admin.s3.amazonaws.com/slot-bb2e601e63b3408bb6a865e442a28366-6f00000002-filestart
<https://onebip-backups-admin.s3.amazonaws.com/slot-bb2e601e63b3408bb6a865e442a28366-6f00000002-filestart>
failed with 404/NoSuchKey
Tue Aug 29 08:27:19 2017: thd-0x1a8a600: amfetchdump: Device
s3:onebip-backups-admin/slot-bb2e601e63b3408bb6a865e442a28366-6 error
= 'Attempt to read past tape-end file'
Tue Aug 29 08:27:19 2017: thd-0x1a8a600: amfetchdump: FAILURE: Attempt
to read past tape-end file
Tue Aug 29 08:27:19 2017: thd-0x1a8a600: amfetchdump: pid 26587 finish
time Tue Aug 29 08:27:19 2017
> Can you get a list of all objects in the S3 bucket?
Here it is. I have scraped the web page and yes the key is missing but
I don’t know why. I mean, I receive the reports every day and they
seemed OK to me. Let me know if you need my conf, as well, thank you.
slot-bb2e601e63b3408bb6a865e442a28366-10f00000001-b0000000000000000.data
slot-bb2e601e63b3408bb6a865e442a28366-10f00000001-filestart
slot-bb2e601e63b3408bb6a865e442a28366-10f00000002-b0000000000000000.data
slot-bb2e601e63b3408bb6a865e442a28366-10f00000002-filestart
slot-bb2e601e63b3408bb6a865e442a28366-10f00000003-b0000000000000000.data
slot-bb2e601e63b3408bb6a865e442a28366-10f00000003-b0000000000000001.data
slot-bb2e601e63b3408bb6a865e442a28366-10f00000003-b0000000000000002.data
slot-bb2e601e63b3408bb6a865e442a28366-10f00000003-b0000000000000003.data
slot-bb2e601e63b3408bb6a865e442a28366-10f00000003-b0000000000000004.data
slot-bb2e601e63b3408bb6a865e442a28366-10f00000003-b0000000000000005.data
slot-bb2e601e63b3408bb6a865e442a28366-10f00000003-b0000000000000006.data
slot-bb2e601e63b3408bb6a865e442a28366-10f00000003-b0000000000000007.data
slot-bb2e601e63b3408bb6a865e442a28366-10f00000003-b0000000000000008.data
slot-bb2e601e63b3408bb6a865e442a28366-10f00000003-b0000000000000009.data
slot-bb2e601e63b3408bb6a865e442a28366-10f00000003-b000000000000000a.data
slot-bb2e601e63b3408bb6a865e442a28366-10f00000003-b000000000000000b.data
slot-bb2e601e63b3408bb6a865e442a28366-10f00000003-b000000000000000c.data
slot-bb2e601e63b3408bb6a865e442a28366-10f00000003-b000000000000000d.data
slot-bb2e601e63b3408bb6a865e442a28366-10f00000003-b000000000000000e.data
slot-bb2e601e63b3408bb6a865e442a28366-10f00000003-b000000000000000f.data
slot-bb2e601e63b3408bb6a865e442a28366-10f00000003-b0000000000000010.data
slot-bb2e601e63b3408bb6a865e442a28366-10f00000003-b0000000000000011.data
slot-bb2e601e63b3408bb6a865e442a28366-10f00000003-b0000000000000012.data
slot-bb2e601e63b3408bb6a865e442a28366-10f00000003-b0000000000000013.data
slot-bb2e601e63b3408bb6a865e442a28366-10f00000003-b0000000000000014.data
slot-bb2e601e63b3408bb6a865e442a28366-10f00000003-b0000000000000015.data
slot-bb2e601e63b3408bb6a865e442a28366-10f00000003-b0000000000000016.data
slot-bb2e601e63b3408bb6a865e442a28366-10f00000003-b0000000000000017.data
slot-bb2e601e63b3408bb6a865e442a28366-10f00000003-b0000000000000018.data
slot-bb2e601e63b3408bb6a865e442a28366-10f00000003-b0000000000000019.data
slot-bb2e601e63b3408bb6a865e442a28366-10f00000003-b000000000000001a.data
slot-bb2e601e63b3408bb6a865e442a28366-10f00000003-filestart
slot-bb2e601e63b3408bb6a865e442a28366-10f00000004-b0000000000000000.data
slot-bb2e601e63b3408bb6a865e442a28366-10f00000004-filestart
slot-bb2e601e63b3408bb6a865e442a28366-10f00000005-b0000000000000000.data
slot-bb2e601e63b3408bb6a865e442a28366-10f00000005-filestart
slot-bb2e601e63b3408bb6a865e442a28366-10special-tapestart
slot-bb2e601e63b3408bb6a865e442a28366-11f00000001-b0000000000000000.data
slot-bb2e601e63b3408bb6a865e442a28366-11f00000001-filestart
slot-bb2e601e63b3408bb6a865e442a28366-11f00000002-b0000000000000000.data
slot-bb2e601e63b3408bb6a865e442a28366-11f00000002-filestart
slot-bb2e601e63b3408bb6a865e442a28366-11f00000003-b0000000000000000.data
slot-bb2e601e63b3408bb6a865e442a28366-11f00000003-filestart
slot-bb2e601e63b3408bb6a865e442a28366-11f00000004-b0000000000000000.data
slot-bb2e601e63b3408bb6a865e442a28366-11f00000004-filestart
slot-bb2e601e63b3408bb6a865e442a28366-11f00000005-b0000000000000000.data
slot-bb2e601e63b3408bb6a865e442a28366-11f00000005-filestart
slot-bb2e601e63b3408bb6a865e442a28366-11special-tapestart
slot-bb2e601e63b3408bb6a865e442a28366-12f00000001-b0000000000000000.data
slot-bb2e601e63b3408bb6a865e442a28366-12f00000001-filestart
slot-bb2e601e63b3408bb6a865e442a28366-12f00000002-b0000000000000000.data
slot-bb2e601e63b3408bb6a865e442a28366-12f00000002-filestart
slot-bb2e601e63b3408bb6a865e442a28366-12f00000003-b0000000000000000.data
slot-bb2e601e63b3408bb6a865e442a28366-12f00000003-filestart
slot-bb2e601e63b3408bb6a865e442a28366-12f00000004-b0000000000000000.data
slot-bb2e601e63b3408bb6a865e442a28366-12f00000004-filestart
slot-bb2e601e63b3408bb6a865e442a28366-12f00000005-b0000000000000000.data
slot-bb2e601e63b3408bb6a865e442a28366-12f00000005-filestart
slot-bb2e601e63b3408bb6a865e442a28366-12special-tapestart
slot-bb2e601e63b3408bb6a865e442a28366-13f00000001-b0000000000000000.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000001-filestart
slot-bb2e601e63b3408bb6a865e442a28366-13f00000002-b0000000000000000.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000002-filestart
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b0000000000000000.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b0000000000000001.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b0000000000000002.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b0000000000000003.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b0000000000000004.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b0000000000000005.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b0000000000000006.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b0000000000000007.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b0000000000000008.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b0000000000000009.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b000000000000000a.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b000000000000000b.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b000000000000000c.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b000000000000000d.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b000000000000000e.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b000000000000000f.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b0000000000000010.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b0000000000000011.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b0000000000000012.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b0000000000000013.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b0000000000000014.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b0000000000000015.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b0000000000000016.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b0000000000000017.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b0000000000000018.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b0000000000000019.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b000000000000001a.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b000000000000001b.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b000000000000001c.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b000000000000001d.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b000000000000001e.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b000000000000001f.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b0000000000000020.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b0000000000000021.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-b0000000000000022.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000003-filestart
slot-bb2e601e63b3408bb6a865e442a28366-13f00000004-b0000000000000000.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000004-filestart
slot-bb2e601e63b3408bb6a865e442a28366-13f00000005-b0000000000000000.data
slot-bb2e601e63b3408bb6a865e442a28366-13f00000005-filestart
slot-bb2e601e63b3408bb6a865e442a28366-13special-tapestart
slot-bb2e601e63b3408bb6a865e442a28366-14special-tapestart
slot-bb2e601e63b3408bb6a865e442a28366-15special-tapestart
slot-bb2e601e63b3408bb6a865e442a28366-1special-tapestart
slot-bb2e601e63b3408bb6a865e442a28366-2special-tapestart
slot-bb2e601e63b3408bb6a865e442a28366-3special-tapestart
slot-bb2e601e63b3408bb6a865e442a28366-4special-tapestart
slot-bb2e601e63b3408bb6a865e442a28366-5special-tapestart
slot-bb2e601e63b3408bb6a865e442a28366-6special-tapestart
slot-bb2e601e63b3408bb6a865e442a28366-7special-tapestart
slot-bb2e601e63b3408bb6a865e442a28366-8special-tapestart
slot-bb2e601e63b3408bb6a865e442a28366-9special-tapestart
>
> Jean-Louis
>
> On 29/08/17 05:22 AM, Giorgio Valoti wrote:
> > Hi all, I’m testing the restore process with Amanda and I’m
getting this error with amfetchdump:
> >
> > amfetchdump -p -a zfs-dataset ci-master rpool/jenkins 20170822000017
> >
> >> 1 volume(s) needed for restoration
> >> The following volumes are needed: slot-6
> >> ERROR: Attempt to read past tape-end file
> > And here is the list of the relevant backups:
> >
> >> backup@backup-server:~$ amadmin zfs-dataset find --sort D
ci-master rpool/jenkins
> >>
> >> date host disk lv tape or file file part status
> >> 2017-08-29 00:00:17 ci-master rpool/jenkins 2 slot-13 3 1/1 OK
> >> 2017-08-28 00:00:17 ci-master rpool/jenkins 2 slot-12 1 1/1 OK
> >> 2017-08-27 00:00:17 ci-master rpool/jenkins 2 slot-11 1 1/1 OK
> >> 2017-08-26 00:00:17 ci-master rpool/jenkins 1 slot-10 3 1/1 OK
> >> 2017-08-25 00:00:17 ci-master rpool/jenkins 1 slot-9 3 1/1 OK
> >> 2017-08-24 00:00:17 ci-master rpool/jenkins 1 slot-8 3 1/1 OK
> >> 2017-08-23 00:00:17 ci-master rpool/jenkins 1 slot-7 3 1/1 OK
> >> 2017-08-22 00:00:17 ci-master rpool/jenkins 0 slot-6 3 1/1 OK
> >> 2017-08-21 00:00:17 ci-master rpool/jenkins 2 slot-5 1 1/1 OK
> >> 2017-08-20 00:00:17 ci-master rpool/jenkins 2 slot-4 1 1/1 OK
> >> 2017-08-19 00:00:17 ci-master rpool/jenkins 2 slot-3 1 1/1 OK
> >> 2017-08-18 00:00:17 ci-master rpool/jenkins 2 slot-2 1 1/1 OK
> >> 2017-08-17 00:00:17 ci-master rpool/jenkins 1 slot-1 3 1/1 OK
> >> 2017-08-16 00:00:17 ci-master rpool/jenkins 1 slot-15 1 1/1 OK
> >> 2017-08-15 00:00:17 ci-master rpool/jenkins 1 slot-14 1 1/1 OK
> > In other words, I’m trying to fetch the level 0 backup. Also, it
should be noted that I’m using “tapes” with other Amanda clients as well.:
> >
> >> backup@backup-server:~$ amadmin zfs-dataset find --sort D | grep
slot-6
> >> 2017-08-22 00:00:17 backup-server rpool/amanda-conf 1 slot-6 5 1/1 OK
> >> 2017-08-22 00:00:17 backup-server rpool/amanda-state 1 slot-6 4
1/1 OK
> >> 2017-08-22 00:00:17 ci-master rpool/jenkins 0 slot-6 3 1/1 OK
> >> 2017-08-22 00:00:17 vpn-server rpool/openvpn 1 slot-6 2 1/1 OK
> >> 2017-08-22 00:00:17 backup-server rpool/amanda-home 1 slot-6 1 1/1 OK
> > I’m sure I’m missing something here, because this is a real
bummer. Anyone on this list willing to help me, please?
> >
> >
> >
> > --
> > Giorgio Valoti
> >
> >
>
>
> Disclaimer
>
> This message is the property of CARBONITE, INC. and may contain
confidential or privileged information.
>
> If this message has been delivered to you by mistake, then do not
copy or deliver this message to anyone. Instead, destroy it and notify
me by reply e-mail.
>
--
Giorgio Valoti
This message is the property of CARBONITE, INC. and may contain confidential or
privileged information.
If this message has been delivered to you by mistake, then do not copy or
deliver this message to anyone. Instead, destroy it and notify me by reply
e-mail
