Re: amanda and pigz
Stefan, amrestore can't restore if the dump is split on multiple file on tape, is it your case? jean-Louis On 02/19/2014 05:10 AM, Stefan G. Weichinger wrote: Am 18.02.2014 19:30, schrieb Stefan G. Weichinger: $ amrestore --config daily /dev/nst0 hiro pigz Restoring from tape daily04 starting with file 1. amrestore: 1: restoring split dumpfile: date 20140218165332 host hiro.oops.intern disk pigz part 1/UNKNOWN lev 1 comp cust program /bin/tar client_custom_compress /usr/bin/pigz ERROR: /usr/bin/pigz exited with status 2 What about this error? See the debug-file: # cat amrestore.20140218192106.debug Tue Feb 18 19:21:06 2014: thd-0x624e00: amrestore: pid 19168 ruid 87 euid 87 version 3.3.5: start at Tue Feb 18 19:21:06 2014 Tue Feb 18 19:21:06 2014: thd-0x624e00: amrestore: Arguments: --config daily /dev/nst0 hiro pigz Tue Feb 18 19:21:06 2014: thd-0x624e00: amrestore: pid 19168 ruid 87 euid 87 version 3.3.5: rename at Tue Feb 18 19:21:06 2014 Tue Feb 18 19:21:06 2014: thd-0x624e00: amrestore: warning: /dev/nst0 uses deprecated device naming convention; using tape:/dev/nst0 instead. Tue Feb 18 19:21:06 2014: thd-0x624e00: amrestore: warning: /dev/nst0 uses deprecated device naming convention; using tape:/dev/nst0 instead. Tue Feb 18 19:21:06 2014: thd-0x624e00: amrestore: Device is in variable block size Tue Feb 18 19:21:13 2014: thd-0x624e00: amrestore: Starting Xfer@0x11b6870 (XferSourceDevice@0x11b3830 - XferFilterProcess@0x11b5800 - XferDestFd@0x11b5040) Tue Feb 18 19:21:13 2014: thd-0x624e00: amrestore: Final linkage: XferSourceDevice@0x11b3830 -(PULL_BUFFER)- XferElementGlue@0x10fc9e0 -(READFD)- XferFilterProcess@0x11b5800 -(WRITEFD)- XferDestFd@0x11b5040 Tue Feb 18 19:21:13 2014: thd-0x624e00: amrestore: XferFilterProcess@0x11b5800 spawning: '/usr/bin/pigz' '-d' Tue Feb 18 19:21:54 2014: thd-0x11b8050: amrestore: Device /dev/nst0 error = 'EOF' Tue Feb 18 19:21:54 2014: thd-0x624e00: amrestore: XferFilterProcess@0x11b5800: process exited with status 2 Tue Feb 18 19:21:54 2014: thd-0x624e00: amrestore: info: ERROR Tue Feb 18 19:21:54 2014: thd-0x624e00: amrestore: Cancelling Xfer@0x11b6870 (XferSourceDevice@0x11b3830 - XferFilterProcess@0x11b5800 - XferDestFd@0x11b5040) Tue Feb 18 19:22:32 2014: thd-0x624e00: amrestore: pid 19168 finish time Tue Feb 18 19:22:32 2014 The problem is that amrestore stops restoring after one such a DLE. Stefan
Re: amanda and pigz
Am 18.02.2014 19:30, schrieb Stefan G. Weichinger: $ amrestore --config daily /dev/nst0 hiro pigz Restoring from tape daily04 starting with file 1. amrestore: 1: restoring split dumpfile: date 20140218165332 host hiro.oops.intern disk pigz part 1/UNKNOWN lev 1 comp cust program /bin/tar client_custom_compress /usr/bin/pigz ERROR: /usr/bin/pigz exited with status 2 What about this error? See the debug-file: # cat amrestore.20140218192106.debug Tue Feb 18 19:21:06 2014: thd-0x624e00: amrestore: pid 19168 ruid 87 euid 87 version 3.3.5: start at Tue Feb 18 19:21:06 2014 Tue Feb 18 19:21:06 2014: thd-0x624e00: amrestore: Arguments: --config daily /dev/nst0 hiro pigz Tue Feb 18 19:21:06 2014: thd-0x624e00: amrestore: pid 19168 ruid 87 euid 87 version 3.3.5: rename at Tue Feb 18 19:21:06 2014 Tue Feb 18 19:21:06 2014: thd-0x624e00: amrestore: warning: /dev/nst0 uses deprecated device naming convention; using tape:/dev/nst0 instead. Tue Feb 18 19:21:06 2014: thd-0x624e00: amrestore: warning: /dev/nst0 uses deprecated device naming convention; using tape:/dev/nst0 instead. Tue Feb 18 19:21:06 2014: thd-0x624e00: amrestore: Device is in variable block size Tue Feb 18 19:21:13 2014: thd-0x624e00: amrestore: Starting Xfer@0x11b6870 (XferSourceDevice@0x11b3830 - XferFilterProcess@0x11b5800 - XferDestFd@0x11b5040) Tue Feb 18 19:21:13 2014: thd-0x624e00: amrestore: Final linkage: XferSourceDevice@0x11b3830 -(PULL_BUFFER)- XferElementGlue@0x10fc9e0 -(READFD)- XferFilterProcess@0x11b5800 -(WRITEFD)- XferDestFd@0x11b5040 Tue Feb 18 19:21:13 2014: thd-0x624e00: amrestore: XferFilterProcess@0x11b5800 spawning: '/usr/bin/pigz' '-d' Tue Feb 18 19:21:54 2014: thd-0x11b8050: amrestore: Device /dev/nst0 error = 'EOF' Tue Feb 18 19:21:54 2014: thd-0x624e00: amrestore: XferFilterProcess@0x11b5800: process exited with status 2 Tue Feb 18 19:21:54 2014: thd-0x624e00: amrestore: info: ERROR Tue Feb 18 19:21:54 2014: thd-0x624e00: amrestore: Cancelling Xfer@0x11b6870 (XferSourceDevice@0x11b3830 - XferFilterProcess@0x11b5800 - XferDestFd@0x11b5040) Tue Feb 18 19:22:32 2014: thd-0x624e00: amrestore: pid 19168 finish time Tue Feb 18 19:22:32 2014 The problem is that amrestore stops restoring after one such a DLE. Stefan
Re: amanda and pigz
Am 03.02.2014 15:29, schrieb Jean-Louis Martineau: On 01/27/2014 01:35 PM, Stefan G. Weichinger wrote: Do I have to use a specific version of amanda (newer than release X) to make use of pigz? I tested 3.3.6 using pigz as a client-compress-custom options and amrestore can restore it. I don't know exactly what's the problem, so I can't tell you when it was fixed. Sorry to say ... I still see this issue on 2 servers. My own amanda server runs (current official stable) 3.3.5 ... I tested (hostname: hiro, config: daily, DLE: pigz) - When I amrestore the very same DLE from the holdingdisk (before having flushed it) it works: # amrestore --config daily --holding /mnt/amhold/daily/20140218165332/hiro.oops.intern.pigz.1 hiro pigz I get the file: $ file hiro.oops.intern.pigz.20140218165332.1.001 hiro.oops.intern.pigz.20140218165332.1.001: POSIX tar archive (GNU) and it is possible to untar it correctly. I flushed this to tape: $ amflush -b daily hiro pigz and tried to restore it: $ amrestore --config daily /dev/nst0 hiro pigz Restoring from tape daily04 starting with file 1. amrestore: 1: restoring split dumpfile: date 20140218165332 host hiro.oops.intern disk pigz part 1/UNKNOWN lev 1 comp cust program /bin/tar client_custom_compress /usr/bin/pigz ERROR: /usr/bin/pigz exited with status 2 What about this error? The fun part is that I get hiro.oops.intern.pigz.20140218165332.1.001 again! So it might be some small cosmetic issue? Maybe you use another version of pigz, JL? I have pigz 2.3.1 here (gentoo linux). Maybe some other amanda user feels helpful and wants to test something similar on his boxes? Thanks, Stefan
Re: amanda and pigz
Am 03.02.2014 um 15:35 schrieb Jean-Louis Martineau: Newer version have a '-a' option that do not needs human interaction. ah, I see that now, but I still have to move the data-Link to the needed tape? How can I then restore while a backup is running when not using multitape? which is rather fine in running automatically, unattended and with no need of an amanda-Database. As I understand, amrestore so far only has been a wrapper for dd if=file bs=32k skip=1? Why has this simple functionality been changed? It is a lot more that a simple dd. I'm sure it does, detecting in which way it was zipped and so on. As I wrote: No offense :) After having build 3.3.5, I'm still experiencing problems. I'm sorry, maybe I'm just blind :/ /michael -- BOFH excuse #283: Lawn mower blade in your fan need sharpening
Re: amanda and pigz
Am 29.01.2014 15:13, schrieb Michael Müskens: So far running amanda-2.6.1 I used to autorestore via amrestore -p tobak012.intern.backup.muessi.de |tar [option] - ./ which is rather fine in running automatically, unattended and with no need of an amanda-Database. As I understand, amrestore so far only has been a wrapper for dd if=file bs=32k skip=1? Why has this simple functionality been changed? No offense, just wondering :) I just tested this at the customer and we also had to do that dd/tar-piping ... why is amanda able to untar gzip-packed stuff correctly (is it still?) and does not do the same for pigz-packed stuff (which is also gzip-format afaik)? call it a feature request ;-) IMO it should not be necessary to use plain dd etc in case of a restore. As a fallback possibility, yes, but not in the normal use of the backup suite, especially not when in a hurry and/or panic because of some server crash ... opinions welcome, Stefan
Re: amanda and pigz
On 01/27/2014 01:35 PM, Stefan G. Weichinger wrote: Do I have to use a specific version of amanda (newer than release X) to make use of pigz? I tested 3.3.6 using pigz as a client-compress-custom options and amrestore can restore it. I don't know exactly what's the problem, so I can't tell you when it was fixed. Jean-Louis I use it for compressing on a new client but the amrestore on the server has problems to unpack the stuff: amanda@backup /mnt/md4/restore/verify $ amrestore /dev/nst0 Restoring from tape daily03 starting with file 1. amrestore: 1: restoring split dumpfile: date 20140123220010 host main disk virt-backup-IKW_SQL part 1/UNKNOWN lev 2 comp N program /bin/tar amrestore: 2: restoring split dumpfile: date 20140123220010 host main disk 2013 part 1/UNKNOWN lev 0 comp cust program /bin/tar client_custom_compress /usr/bin/pigz ERROR: /usr/bin/pigz exited with status 2 amanda-3.2.1 on the server (we had to keep it on that version because of several dependencies ... app, glibc, kernel) ... pigz installed long after amanda ... in the correct path. So far I haven't found out what status 2 means here. I assume I should re-install amanda with debug on and read the logs, right? Thanks, Stefan
Re: amanda and pigz
On 01/29/2014 09:13 AM, Michael Müskens wrote: You can't use amrestore to directly read the dump from a vtape, use something like: amrestore -r file:/backup/tapepools/DailySet/DailySet-69 For that I would have to point the data-link to the needed tape? That is something I actually wanted to avoid :) or : amtape CONF label DailySet-69 amrestore -r file:/backup/tapepools/DailySet It is a lot easier to use amfetchdump: amfetchdump CONF tobak012.intern.backup.muessi.de /etc 20140128003001 That command obviously needs human interaction and has to be run as user backup? Newer version have a '-a' option that do not needs human interaction. backup@tobak012:~/hoehoe$ amfetchdump DailySet tobak012.intern.backup.muessi.de /etc 20140128 1 volume(s) needed for restoration The following volumes are needed: DailySet-69 Press enter when ready I want to avoid that too. So far running amanda-2.6.1 I used to autorestore via amrestore -p tobak012.intern.backup.muessi.de |tar [option] - ./ which is rather fine in running automatically, unattended and with no need of an amanda-Database. As I understand, amrestore so far only has been a wrapper for dd if=file bs=32k skip=1? Why has this simple functionality been changed? It is a lot more that a simple dd. No offense, just wondering :) /mm -- BOFH excuse #244: Your cat tried to eat the mouse.
Re: amanda and pigz
You can't use amrestore to directly read the dump from a vtape, use something like: amrestore -r file:/backup/tapepools/DailySet/DailySet-69 For that I would have to point the data-link to the needed tape? That is something I actually wanted to avoid :) or : amtape CONF label DailySet-69 amrestore -r file:/backup/tapepools/DailySet It is a lot easier to use amfetchdump: amfetchdump CONF tobak012.intern.backup.muessi.de /etc 20140128003001 That command obviously needs human interaction and has to be run as user backup? backup@tobak012:~/hoehoe$ amfetchdump DailySet tobak012.intern.backup.muessi.de /etc 20140128 1 volume(s) needed for restoration The following volumes are needed: DailySet-69 Press enter when ready I want to avoid that too. So far running amanda-2.6.1 I used to autorestore via amrestore -p tobak012.intern.backup.muessi.de |tar [option] - ./ which is rather fine in running automatically, unattended and with no need of an amanda-Database. As I understand, amrestore so far only has been a wrapper for dd if=file bs=32k skip=1? Why has this simple functionality been changed? No offense, just wondering :) /mm -- BOFH excuse #244: Your cat tried to eat the mouse.
Re: amanda and pigz
Hi guys, Use the '-r' argument of amrestore and manually uncompress the file with pigz or gzip I have a similar Problem with amanda-3.3.1 and with argument -r I get: root@tobak012:~/manual_restore/dussel# amrestore -r /backup/tapepools/DailySet/DailySet-69/3.tobak012.intern.backup.muessi.de._etc.0 Reading from '/backup/tapepools/DailySet/DailySet-69/3.tobak012.intern.backup.muessi.de._etc.0' split dumpfile: date 20140128003001 host tobak012.intern.backup.muessi.de disk /etc part 1/UNKNOWN lev 0 comp .gz program /bin/tar ERROR: unexpected header type 5 in holding file '/backup/tapepools/DailySet/DailySet-69/3.tobak012.intern.backup.muessi.de._etc.0' Extracting the dump via dd is working fine... Any suggestions? TIA Michael -- If you can put an axe through it, it's hardware; if you *want* to put an axe through it, it's software.
Re: amanda and pigz
On 01/28/2014 09:00 AM, Michael Müskens wrote: Hi guys, Use the '-r' argument of amrestore and manually uncompress the file with pigz or gzip I have a similar Problem with amanda-3.3.1 and with argument -r I get: root@tobak012:~/manual_restore/dussel# amrestore -r /backup/tapepools/DailySet/DailySet-69/3.tobak012.intern.backup.muessi.de._etc.0 You can't use amrestore to directly read the dump from a vtape, use something like: amrestore -r file:/backup/tapepools/DailySet/DailySet-69 or : amtape CONF label DailySet-69 amrestore -r file:/backup/tapepools/DailySet It is a lot easier to use amfetchdump: amfetchdump CONF tobak012.intern.backup.muessi.de /etc 20140128003001 Jean-Louis Reading from '/backup/tapepools/DailySet/DailySet-69/3.tobak012.intern.backup.muessi.de._etc.0' split dumpfile: date 20140128003001 host tobak012.intern.backup.muessi.de disk /etc part 1/UNKNOWN lev 0 comp .gz program /bin/tar ERROR: unexpected header type 5 in holding file '/backup/tapepools/DailySet/DailySet-69/3.tobak012.intern.backup.muessi.de._etc.0' Extracting the dump via dd is working fine... Any suggestions? TIA Michael -- If you can put an axe through it, it's hardware; if you *want* to put an axe through it, it's software.
amanda and pigz
Do I have to use a specific version of amanda (newer than release X) to make use of pigz? I use it for compressing on a new client but the amrestore on the server has problems to unpack the stuff: amanda@backup /mnt/md4/restore/verify $ amrestore /dev/nst0 Restoring from tape daily03 starting with file 1. amrestore: 1: restoring split dumpfile: date 20140123220010 host main disk virt-backup-IKW_SQL part 1/UNKNOWN lev 2 comp N program /bin/tar amrestore: 2: restoring split dumpfile: date 20140123220010 host main disk 2013 part 1/UNKNOWN lev 0 comp cust program /bin/tar client_custom_compress /usr/bin/pigz ERROR: /usr/bin/pigz exited with status 2 amanda-3.2.1 on the server (we had to keep it on that version because of several dependencies ... app, glibc, kernel) ... pigz installed long after amanda ... in the correct path. So far I haven't found out what status 2 means here. I assume I should re-install amanda with debug on and read the logs, right? Thanks, Stefan
Re: amanda and pigz
Use the '-r' argument of amrestore and manually uncompress the file with pigz or gzip Jean-Louis On 01/27/2014 01:35 PM, Stefan G. Weichinger wrote: Do I have to use a specific version of amanda (newer than release X) to make use of pigz? I use it for compressing on a new client but the amrestore on the server has problems to unpack the stuff: amanda@backup /mnt/md4/restore/verify $ amrestore /dev/nst0 Restoring from tape daily03 starting with file 1. amrestore: 1: restoring split dumpfile: date 20140123220010 host main disk virt-backup-IKW_SQL part 1/UNKNOWN lev 2 comp N program /bin/tar amrestore: 2: restoring split dumpfile: date 20140123220010 host main disk 2013 part 1/UNKNOWN lev 0 comp cust program /bin/tar client_custom_compress /usr/bin/pigz ERROR: /usr/bin/pigz exited with status 2 amanda-3.2.1 on the server (we had to keep it on that version because of several dependencies ... app, glibc, kernel) ... pigz installed long after amanda ... in the correct path. So far I haven't found out what status 2 means here. I assume I should re-install amanda with debug on and read the logs, right? Thanks, Stefan
