Re: Duplicity & /etc/daily.local
On 22/05/2019 04:46, Patrick Wildt wrote: On Mon, May 20, 2019 at 11:50:13PM +0200, Noth wrote: Hi misc@, I'm trying to run daily backups to a sftp server for various VMs and devices on my network, and want to use /etc/daily.local for this. I'm calling this script from the daily.local file: env 'GNUPG="/usr/local/bin/gpg" PASSPHRASE="mypassword"' /root/duplicity-hostname.sh but unfortunately duplicity can't find gnupg and errors out with this error message: Traceback (innermost last): File "/usr/local/bin/duplicity", line 1562, in with_tempdir(main) File "/usr/local/bin/duplicity", line 1548, in with_tempdir fn() File "/usr/local/bin/duplicity", line 1387, in main action = commandline.ProcessCommandLine(sys.argv[1:]) File "/usr/local/lib/python2.7/site-packages/duplicity/commandline.py", line 1088, in ProcessCommandLine globals.gpg_profile = gpg.GPGProfile() File "/usr/local/lib/python2.7/site-packages/duplicity/gpg.py", line 92, in __init__ self.gpg_version = self.get_gpg_version(globals.gpg_binary) File "/usr/local/lib/python2.7/site-packages/duplicity/gpg.py", line 107, in get_gpg_version res = gnupg.run(["--version"], create_fhs=["stdout"]) File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", line 374, in run create_fhs, attach_fhs) File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", line 423, in _attach_fork_exec self._as_child(process, gnupg_commands, args) File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", line 462, in _as_child os.execvp(command[0], command) File "/usr/local/lib/python2.7/os.py", line 346, in execvp _execvpe(file, args) File "/usr/local/lib/python2.7/os.py", line 382, in _execvpe func(fullname, *argrest) OSError: [Errno 2] No such file or directory GPGError: failed to determine gnupg version of None from duplicity-hostname.sh content: #!/bin/ksh PASSPHRASE=mypassword /usr/local/bin/duplicity incremental /var sftp://user@backuphost:/hostname/var /usr/local/bin/duplicity incremental /etc sftp://user@backuphost:/hostname/etc /usr/local/bin/duplicity incremental /root sftp://user@backuphost:/hostname/root Can daily.local even handle this or is the environment too limited? Cheers, Noth I have the same setup and it failed for me as well. I somehow managed to fix it by setting PATH and also exporting TERM: PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/X11R6/bin:/usr/local/bin:/usr/local/sbin:/usr/games TERM=xterm export PATH TERM And you should probably also do something like: . /root/.passphrase test -n "$PASSPHRASE" || exit 0 export PASSPHRASE Patrick Hello again, Sorry about the delay, I haven't been able to look at this till today. So: 1. adding PATH does make duplicity find gpg. Thanks Antoine Jacoutot for the suggestion on getting the env mailed via cron. 2. Next issue is that the sftp connection fails, because it can't authenticate the host and wants confirmation on accepting the key. This is despite having copied the user public key to user/.ssh/.authorized_keys and tested using sftp user@hostname and sftp user@hostname.domain . Current state of the script is this: #!/bin/ksh export PASSPHRASE="mypassword" export PATH=${PATH}:/usr/local/bin /usr/local/bin/duplicity incremental --ssh-options="-oStrictHostKeyChecking=no" /var sftp://user@backuphost:/hostname/var As you can see I'm passing the option so it doesn't ask for confirmation, but it still happens: /usr/local/lib/python2.7/site-packages/paramiko/kex_ecdh_nist.py:39: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding. m.add_string(self.Q_C.public_numbers().encode_point()) /usr/local/lib/python2.7/site-packages/paramiko/kex_ecdh_nist.py:92: CryptographyDeprecationWarning: Support for unsafe construction of public numbers from encoded data will be removed in a future version. Please use EllipticCurvePublicKey.from_encoded_point self.curve, Q_S_bytes /usr/local/lib/python2.7/site-packages/paramiko/kex_ecdh_nist.py:103: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding. hm.add_string(self.Q_C.public_numbers().encode_point()) BackendException: ssh connection to dupuser@tombstone:22 failed: EOF when reading a line The authenticity of host 'tombstone' can't be established. SSH-ED25519 key fingerprint is fd:60:79:e6:d9:06:47:41:b6:54:19:63:9c:b0:d2:89. Are you sure you want to continue connecting (yes/no)? /usr/local/lib/python2.7/site-packages/paramiko I tried adding StrictHostKeyChecking no to /etc/ssh/ssh_config and /root/.ssh/config
Re: Duplicity & /etc/daily.local
On Mon, May 20, 2019 at 11:50:13PM +0200, Noth wrote: > Hi misc@, > > > I'm trying to run daily backups to a sftp server for various VMs and > devices on my network, and want to use /etc/daily.local for this. I'm > calling this script from the daily.local file: > > env 'GNUPG="/usr/local/bin/gpg" PASSPHRASE="mypassword"' > /root/duplicity-hostname.sh > > but unfortunately duplicity can't find gnupg and errors out with this error > message: > > Traceback (innermost last): > File "/usr/local/bin/duplicity", line 1562, in > with_tempdir(main) > File "/usr/local/bin/duplicity", line 1548, in with_tempdir > fn() > File "/usr/local/bin/duplicity", line 1387, in main > action = commandline.ProcessCommandLine(sys.argv[1:]) > File "/usr/local/lib/python2.7/site-packages/duplicity/commandline.py", > line 1088, in ProcessCommandLine > globals.gpg_profile = gpg.GPGProfile() > File "/usr/local/lib/python2.7/site-packages/duplicity/gpg.py", line 92, in > __init__ > self.gpg_version = self.get_gpg_version(globals.gpg_binary) > File "/usr/local/lib/python2.7/site-packages/duplicity/gpg.py", line 107, > in get_gpg_version > res = gnupg.run(["--version"], create_fhs=["stdout"]) > File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", > line 374, in run > create_fhs, attach_fhs) > File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", > line 423, in _attach_fork_exec > self._as_child(process, gnupg_commands, args) > File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", > line 462, in _as_child > os.execvp(command[0], command) > File "/usr/local/lib/python2.7/os.py", line 346, in execvp > _execvpe(file, args) > File "/usr/local/lib/python2.7/os.py", line 382, in _execvpe > func(fullname, *argrest) > OSError: [Errno 2] No such file or directory > > GPGError: failed to determine gnupg version of None from > > > duplicity-hostname.sh content: > > #!/bin/ksh > PASSPHRASE=mypassword > /usr/local/bin/duplicity incremental /var sftp://user@backuphost:/hostname/var > /usr/local/bin/duplicity incremental /etc sftp://user@backuphost:/hostname/etc > /usr/local/bin/duplicity incremental /root > sftp://user@backuphost:/hostname/root > > Can daily.local even handle this or is the environment too limited? > > Cheers, > > Noth > I have the same setup and it failed for me as well. I somehow managed to fix it by setting PATH and also exporting TERM: PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/X11R6/bin:/usr/local/bin:/usr/local/sbin:/usr/games TERM=xterm export PATH TERM And you should probably also do something like: . /root/.passphrase test -n "$PASSPHRASE" || exit 0 export PASSPHRASE Patrick
Re: Duplicity & /etc/daily.local
Hi Noth, On Mon, 20 May 2019 23:50:13 +0200 Noth wrote: > /root/duplicity-hostname.sh Does this script work properly when you're logged in as root? > > #!/bin/ksh > PASSPHRASE=mypassword try: export PASSPHRASE='mypassword' > Can daily.local even handle this or is the environment too limited? daily is run by root's crontab. To get root's cron environment mailed out, add a temp root cron job: [next 5 mins] [this hour] * * * logname; umask; pwd; printenv | sort If your script works well when logged in as root, but not from cron, add the missing environment elements to daily.local/script/root's crontab. Cheers, -- Craig Skinner | http://linkd.in/yGqkv7
Re: Duplicity & /etc/daily.local
On Mon, May 20, 2019 at 11:50:13PM +0200, Noth wrote: > Hi misc@, > > > I'm trying to run daily backups to a sftp server for various VMs and > devices on my network, and want to use /etc/daily.local for this. I'm > calling this script from the daily.local file: > > env 'GNUPG="/usr/local/bin/gpg" PASSPHRASE="mypassword"' > /root/duplicity-hostname.sh > > but unfortunately duplicity can't find gnupg and errors out with this error > message: > > Traceback (innermost last): > File "/usr/local/bin/duplicity", line 1562, in > with_tempdir(main) > File "/usr/local/bin/duplicity", line 1548, in with_tempdir > fn() > File "/usr/local/bin/duplicity", line 1387, in main > action = commandline.ProcessCommandLine(sys.argv[1:]) > File "/usr/local/lib/python2.7/site-packages/duplicity/commandline.py", > line 1088, in ProcessCommandLine > globals.gpg_profile = gpg.GPGProfile() > File "/usr/local/lib/python2.7/site-packages/duplicity/gpg.py", line 92, in > __init__ > self.gpg_version = self.get_gpg_version(globals.gpg_binary) > File "/usr/local/lib/python2.7/site-packages/duplicity/gpg.py", line 107, > in get_gpg_version > res = gnupg.run(["--version"], create_fhs=["stdout"]) > File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", > line 374, in run > create_fhs, attach_fhs) > File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", > line 423, in _attach_fork_exec > self._as_child(process, gnupg_commands, args) > File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", > line 462, in _as_child > os.execvp(command[0], command) > File "/usr/local/lib/python2.7/os.py", line 346, in execvp > _execvpe(file, args) > File "/usr/local/lib/python2.7/os.py", line 382, in _execvpe > func(fullname, *argrest) > OSError: [Errno 2] No such file or directory > > GPGError: failed to determine gnupg version of None from > > > duplicity-hostname.sh content: > > #!/bin/ksh > PASSPHRASE=mypassword > /usr/local/bin/duplicity incremental /var sftp://user@backuphost:/hostname/var > /usr/local/bin/duplicity incremental /etc sftp://user@backuphost:/hostname/etc > /usr/local/bin/duplicity incremental /root > sftp://user@backuphost:/hostname/root > > Can daily.local even handle this or is the environment too limited? daily.local is run by cron which sets: PATH=/bin:/sbin:/usr/bin:/usr/sbin Try setting this in your script: PATH=${PATH}:/usr/local/bin -- Antoine
Re: Duplicity & /etc/daily.local
On Mon, May 20, 2019 5:50 pm, Noth wrote: > Hi misc@, > > > Â I'm trying to run daily backups to a sftp server for various VMs and > devices on my network, and want to use /etc/daily.local for this. I'm > calling this script from the daily.local file: > > env 'GNUPG="/usr/local/bin/gpg" PASSPHRASE="mypassword"' > /root/duplicity-hostname.sh > > but unfortunately duplicity can't find gnupg > I don't use duplicity anymore but is the GNUPG environment variable even a thing? Their manpage doesn't mention it but does specifiy a --ggp-binary commandline argument. http://duplicity.nongnu.org/duplicity.1.html You'll also need to be sure gpg is looking in the right place for the keyrings.
Duplicity & /etc/daily.local
Hi misc@, I'm trying to run daily backups to a sftp server for various VMs and devices on my network, and want to use /etc/daily.local for this. I'm calling this script from the daily.local file: env 'GNUPG="/usr/local/bin/gpg" PASSPHRASE="mypassword"' /root/duplicity-hostname.sh but unfortunately duplicity can't find gnupg and errors out with this error message: Traceback (innermost last): File "/usr/local/bin/duplicity", line 1562, in with_tempdir(main) File "/usr/local/bin/duplicity", line 1548, in with_tempdir fn() File "/usr/local/bin/duplicity", line 1387, in main action = commandline.ProcessCommandLine(sys.argv[1:]) File "/usr/local/lib/python2.7/site-packages/duplicity/commandline.py", line 1088, in ProcessCommandLine globals.gpg_profile = gpg.GPGProfile() File "/usr/local/lib/python2.7/site-packages/duplicity/gpg.py", line 92, in __init__ self.gpg_version = self.get_gpg_version(globals.gpg_binary) File "/usr/local/lib/python2.7/site-packages/duplicity/gpg.py", line 107, in get_gpg_version res = gnupg.run(["--version"], create_fhs=["stdout"]) File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", line 374, in run create_fhs, attach_fhs) File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", line 423, in _attach_fork_exec self._as_child(process, gnupg_commands, args) File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", line 462, in _as_child os.execvp(command[0], command) File "/usr/local/lib/python2.7/os.py", line 346, in execvp _execvpe(file, args) File "/usr/local/lib/python2.7/os.py", line 382, in _execvpe func(fullname, *argrest) OSError: [Errno 2] No such file or directory GPGError: failed to determine gnupg version of None from duplicity-hostname.sh content: #!/bin/ksh PASSPHRASE=mypassword /usr/local/bin/duplicity incremental /var sftp://user@backuphost:/hostname/var /usr/local/bin/duplicity incremental /etc sftp://user@backuphost:/hostname/etc /usr/local/bin/duplicity incremental /root sftp://user@backuphost:/hostname/root Can daily.local even handle this or is the environment too limited? Cheers, Noth