Re: Duplicity & /etc/daily.local

2019-05-23 Thread Noth 



On 22/05/2019 04:46, Patrick Wildt wrote:

On Mon, May 20, 2019 at 11:50:13PM +0200, Noth wrote:

Hi misc@,


   I'm trying to run daily backups to a sftp server for various VMs and
devices on my network, and want to use /etc/daily.local for this. I'm
calling this script from the daily.local file:

env 'GNUPG="/usr/local/bin/gpg" PASSPHRASE="mypassword"'
/root/duplicity-hostname.sh

but unfortunately duplicity can't find gnupg and errors out with this error
message:

Traceback (innermost last):
   File "/usr/local/bin/duplicity", line 1562, in 
 with_tempdir(main)
   File "/usr/local/bin/duplicity", line 1548, in with_tempdir
 fn()
   File "/usr/local/bin/duplicity", line 1387, in main
 action = commandline.ProcessCommandLine(sys.argv[1:])
   File "/usr/local/lib/python2.7/site-packages/duplicity/commandline.py", line 
1088, in ProcessCommandLine
 globals.gpg_profile = gpg.GPGProfile()
   File "/usr/local/lib/python2.7/site-packages/duplicity/gpg.py", line 92, in 
__init__
 self.gpg_version = self.get_gpg_version(globals.gpg_binary)
   File "/usr/local/lib/python2.7/site-packages/duplicity/gpg.py", line 107, in 
get_gpg_version
 res = gnupg.run(["--version"], create_fhs=["stdout"])
   File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", 
line 374, in run
 create_fhs, attach_fhs)
   File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", 
line 423, in _attach_fork_exec
 self._as_child(process, gnupg_commands, args)
   File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", 
line 462, in _as_child
 os.execvp(command[0], command)
   File "/usr/local/lib/python2.7/os.py", line 346, in execvp
 _execvpe(file, args)
   File "/usr/local/lib/python2.7/os.py", line 382, in _execvpe
 func(fullname, *argrest)
  OSError: [Errno 2] No such file or directory

GPGError: failed to determine gnupg version of None from


duplicity-hostname.sh content:

#!/bin/ksh
PASSPHRASE=mypassword
/usr/local/bin/duplicity incremental /var sftp://user@backuphost:/hostname/var
/usr/local/bin/duplicity incremental /etc sftp://user@backuphost:/hostname/etc
/usr/local/bin/duplicity incremental /root sftp://user@backuphost:/hostname/root

Can daily.local even handle this or is the environment too limited?

Cheers,

Noth


I have the same setup and it failed for me as well.  I somehow managed
to fix it by setting PATH and also exporting TERM:

PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/X11R6/bin:/usr/local/bin:/usr/local/sbin:/usr/games
TERM=xterm
export PATH TERM

And you should probably also do something like:

. /root/.passphrase
test -n "$PASSPHRASE" || exit 0
export PASSPHRASE

Patrick


Hello again,

  Sorry about the delay, I haven't been able to look at this till today.

So:

1. adding PATH does make duplicity find gpg. Thanks Antoine Jacoutot for 
the suggestion on getting the env mailed via cron.


2. Next issue is that the sftp connection fails, because it can't 
authenticate the host and wants confirmation on accepting the key. This 
is despite having copied the user public key to 
user/.ssh/.authorized_keys and tested using sftp user@hostname and sftp 
user@hostname.domain . Current state of the script is this:



#!/bin/ksh
export PASSPHRASE="mypassword"
export PATH=${PATH}:/usr/local/bin
/usr/local/bin/duplicity incremental 
--ssh-options="-oStrictHostKeyChecking=no" /var 
sftp://user@backuphost:/hostname/var


As you can see I'm passing the option so it doesn't ask for 
confirmation, but it still happens:



/usr/local/lib/python2.7/site-packages/paramiko/kex_ecdh_nist.py:39: 
CryptographyDeprecationWarning: encode_point has been deprecated on 
EllipticCurvePublicNumbers and will be removed in a future version. Please use 
EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed 
point encoding.
  m.add_string(self.Q_C.public_numbers().encode_point())
/usr/local/lib/python2.7/site-packages/paramiko/kex_ecdh_nist.py:92: 
CryptographyDeprecationWarning: Support for unsafe construction of public 
numbers from encoded data will be removed in a future version. Please use 
EllipticCurvePublicKey.from_encoded_point
  self.curve, Q_S_bytes
/usr/local/lib/python2.7/site-packages/paramiko/kex_ecdh_nist.py:103: 
CryptographyDeprecationWarning: encode_point has been deprecated on 
EllipticCurvePublicNumbers and will be removed in a future version. Please use 
EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed 
point encoding.
  hm.add_string(self.Q_C.public_numbers().encode_point())
BackendException: ssh connection to dupuser@tombstone:22 failed: EOF when 
reading a line
The authenticity of host 'tombstone' can't be established.
SSH-ED25519 key fingerprint is fd:60:79:e6:d9:06:47:41:b6:54:19:63:9c:b0:d2:89.
Are you sure you want to continue connecting (yes/no)? 
/usr/local/lib/python2.7/site-packages/paramiko


I tried adding StrictHostKeyChecking no to /etc/ssh/ssh_config and

Re: Duplicity & /etc/daily.local

2019-05-21 Thread Patrick Wildt 
On Mon, May 20, 2019 at 11:50:13PM +0200, Noth wrote:
> Hi misc@,
> 
> 
>   I'm trying to run daily backups to a sftp server for various VMs and
> devices on my network, and want to use /etc/daily.local for this. I'm
> calling this script from the daily.local file:
> 
> env 'GNUPG="/usr/local/bin/gpg" PASSPHRASE="mypassword"'
> /root/duplicity-hostname.sh
> 
> but unfortunately duplicity can't find gnupg and errors out with this error
> message:
> 
> Traceback (innermost last):
>   File "/usr/local/bin/duplicity", line 1562, in 
> with_tempdir(main)
>   File "/usr/local/bin/duplicity", line 1548, in with_tempdir
> fn()
>   File "/usr/local/bin/duplicity", line 1387, in main
> action = commandline.ProcessCommandLine(sys.argv[1:])
>   File "/usr/local/lib/python2.7/site-packages/duplicity/commandline.py", 
> line 1088, in ProcessCommandLine
> globals.gpg_profile = gpg.GPGProfile()
>   File "/usr/local/lib/python2.7/site-packages/duplicity/gpg.py", line 92, in 
> __init__
> self.gpg_version = self.get_gpg_version(globals.gpg_binary)
>   File "/usr/local/lib/python2.7/site-packages/duplicity/gpg.py", line 107, 
> in get_gpg_version
> res = gnupg.run(["--version"], create_fhs=["stdout"])
>   File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", 
> line 374, in run
> create_fhs, attach_fhs)
>   File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", 
> line 423, in _attach_fork_exec
> self._as_child(process, gnupg_commands, args)
>   File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", 
> line 462, in _as_child
> os.execvp(command[0], command)
>   File "/usr/local/lib/python2.7/os.py", line 346, in execvp
> _execvpe(file, args)
>   File "/usr/local/lib/python2.7/os.py", line 382, in _execvpe
> func(fullname, *argrest)
>  OSError: [Errno 2] No such file or directory
> 
> GPGError: failed to determine gnupg version of None from
> 
> 
> duplicity-hostname.sh content:
> 
> #!/bin/ksh
> PASSPHRASE=mypassword
> /usr/local/bin/duplicity incremental /var sftp://user@backuphost:/hostname/var
> /usr/local/bin/duplicity incremental /etc sftp://user@backuphost:/hostname/etc
> /usr/local/bin/duplicity incremental /root 
> sftp://user@backuphost:/hostname/root
> 
> Can daily.local even handle this or is the environment too limited?
> 
> Cheers,
> 
> Noth
> 

I have the same setup and it failed for me as well.  I somehow managed
to fix it by setting PATH and also exporting TERM:

PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/X11R6/bin:/usr/local/bin:/usr/local/sbin:/usr/games
TERM=xterm
export PATH TERM

And you should probably also do something like:

. /root/.passphrase
test -n "$PASSPHRASE" || exit 0
export PASSPHRASE

Patrick

Re: Duplicity & /etc/daily.local

2019-05-21 Thread Craig Skinner 
Hi Noth,

On Mon, 20 May 2019 23:50:13 +0200 Noth wrote:
> /root/duplicity-hostname.sh

Does this script work properly when you're logged in as root?


> 
> #!/bin/ksh
> PASSPHRASE=mypassword

try: export PASSPHRASE='mypassword'


> Can daily.local even handle this or is the environment too limited?



daily is run by root's crontab.

To get root's cron environment mailed out, add a temp root cron job:

[next 5 mins] [this hour]  *  *   *  logname; umask; pwd; printenv | sort


If your script works well when logged in as root, but not from cron,
add the missing environment elements to daily.local/script/root's crontab.


Cheers,
-- 
Craig Skinner | http://linkd.in/yGqkv7

Re: Duplicity & /etc/daily.local

2019-05-21 Thread Antoine Jacoutot 
On Mon, May 20, 2019 at 11:50:13PM +0200, Noth wrote:
> Hi misc@,
> 
> 
>   I'm trying to run daily backups to a sftp server for various VMs and
> devices on my network, and want to use /etc/daily.local for this. I'm
> calling this script from the daily.local file:
> 
> env 'GNUPG="/usr/local/bin/gpg" PASSPHRASE="mypassword"'
> /root/duplicity-hostname.sh
> 
> but unfortunately duplicity can't find gnupg and errors out with this error
> message:
> 
> Traceback (innermost last):
>   File "/usr/local/bin/duplicity", line 1562, in 
> with_tempdir(main)
>   File "/usr/local/bin/duplicity", line 1548, in with_tempdir
> fn()
>   File "/usr/local/bin/duplicity", line 1387, in main
> action = commandline.ProcessCommandLine(sys.argv[1:])
>   File "/usr/local/lib/python2.7/site-packages/duplicity/commandline.py", 
> line 1088, in ProcessCommandLine
> globals.gpg_profile = gpg.GPGProfile()
>   File "/usr/local/lib/python2.7/site-packages/duplicity/gpg.py", line 92, in 
> __init__
> self.gpg_version = self.get_gpg_version(globals.gpg_binary)
>   File "/usr/local/lib/python2.7/site-packages/duplicity/gpg.py", line 107, 
> in get_gpg_version
> res = gnupg.run(["--version"], create_fhs=["stdout"])
>   File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", 
> line 374, in run
> create_fhs, attach_fhs)
>   File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", 
> line 423, in _attach_fork_exec
> self._as_child(process, gnupg_commands, args)
>   File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", 
> line 462, in _as_child
> os.execvp(command[0], command)
>   File "/usr/local/lib/python2.7/os.py", line 346, in execvp
> _execvpe(file, args)
>   File "/usr/local/lib/python2.7/os.py", line 382, in _execvpe
> func(fullname, *argrest)
>  OSError: [Errno 2] No such file or directory
> 
> GPGError: failed to determine gnupg version of None from
> 
> 
> duplicity-hostname.sh content:
> 
> #!/bin/ksh
> PASSPHRASE=mypassword
> /usr/local/bin/duplicity incremental /var sftp://user@backuphost:/hostname/var
> /usr/local/bin/duplicity incremental /etc sftp://user@backuphost:/hostname/etc
> /usr/local/bin/duplicity incremental /root 
> sftp://user@backuphost:/hostname/root
> 
> Can daily.local even handle this or is the environment too limited?

daily.local is run by cron which sets:
PATH=/bin:/sbin:/usr/bin:/usr/sbin

Try setting this in your script:
PATH=${PATH}:/usr/local/bin

-- 
Antoine

Re: Duplicity & /etc/daily.local

2019-05-20 Thread trondd 
On Mon, May 20, 2019 5:50 pm, Noth wrote:
> Hi misc@,
>
>
>  Â I'm trying to run daily backups to a sftp server for various VMs and
> devices on my network, and want to use /etc/daily.local for this. I'm
> calling this script from the daily.local file:
>
> env 'GNUPG="/usr/local/bin/gpg" PASSPHRASE="mypassword"'
> /root/duplicity-hostname.sh
>
> but unfortunately duplicity can't find gnupg
>

I don't use duplicity anymore but is the GNUPG environment variable even a
thing?  Their manpage doesn't mention it but does specifiy a --ggp-binary
commandline argument.

http://duplicity.nongnu.org/duplicity.1.html

You'll also need to be sure gpg is looking in the right place for the
keyrings.

Duplicity & /etc/daily.local

2019-05-20 Thread Noth 

Hi misc@,


  I'm trying to run daily backups to a sftp server for various VMs and 
devices on my network, and want to use /etc/daily.local for this. I'm 
calling this script from the daily.local file:


env 'GNUPG="/usr/local/bin/gpg" PASSPHRASE="mypassword"' 
/root/duplicity-hostname.sh


but unfortunately duplicity can't find gnupg and errors out with this 
error message:


Traceback (innermost last):
  File "/usr/local/bin/duplicity", line 1562, in 
with_tempdir(main)
  File "/usr/local/bin/duplicity", line 1548, in with_tempdir
fn()
  File "/usr/local/bin/duplicity", line 1387, in main
action = commandline.ProcessCommandLine(sys.argv[1:])
  File "/usr/local/lib/python2.7/site-packages/duplicity/commandline.py", line 
1088, in ProcessCommandLine
globals.gpg_profile = gpg.GPGProfile()
  File "/usr/local/lib/python2.7/site-packages/duplicity/gpg.py", line 92, in 
__init__
self.gpg_version = self.get_gpg_version(globals.gpg_binary)
  File "/usr/local/lib/python2.7/site-packages/duplicity/gpg.py", line 107, in 
get_gpg_version
res = gnupg.run(["--version"], create_fhs=["stdout"])
  File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", line 
374, in run
create_fhs, attach_fhs)
  File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", line 
423, in _attach_fork_exec
self._as_child(process, gnupg_commands, args)
  File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", line 
462, in _as_child
os.execvp(command[0], command)
  File "/usr/local/lib/python2.7/os.py", line 346, in execvp
_execvpe(file, args)
  File "/usr/local/lib/python2.7/os.py", line 382, in _execvpe
func(fullname, *argrest)
 OSError: [Errno 2] No such file or directory

GPGError: failed to determine gnupg version of None from


duplicity-hostname.sh content:

#!/bin/ksh
PASSPHRASE=mypassword
/usr/local/bin/duplicity incremental /var sftp://user@backuphost:/hostname/var
/usr/local/bin/duplicity incremental /etc sftp://user@backuphost:/hostname/etc
/usr/local/bin/duplicity incremental /root sftp://user@backuphost:/hostname/root

Can daily.local even handle this or is the environment too limited?

Cheers,

Noth