Hello community, here is the log from the commit of package duply for openSUSE:Factory checked in at 2016-09-30 15:33:10 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/duply (Old) and /work/SRC/openSUSE:Factory/.duply.new (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "duply" Changes: -------- --- /work/SRC/openSUSE:Factory/duply/duply.changes 2015-10-02 09:23:37.000000000 +0200 +++ /work/SRC/openSUSE:Factory/.duply.new/duply.changes 2016-09-30 15:33:15.000000000 +0200 @@ -1,0 +2,31 @@ +Mon Sep 26 01:42:40 UTC 2016 - [email protected] + +- Update to version 1.11.3: + + Fix wrong "WARNING: No running gpg-agent ..." when sign key was + not set. +- Changes from version 1.11.2: + + Fix "gpg: unsafe" version print out. + + Bugfix 91: v1.11 [r47] broke asymmetric encryption when using + GPG_KEYS_ENC. + + Bugfix 90: S3: TARGET_USER/PASS have no effect, added + additional documentation about needed env vars to template + conf file. +- Changes from version 1.11.1: + + Reverted duply's default PYTHON to 'python'. +- Changes from version 1.11: + + Remove obsolete --ssh-askpass routine. + + Add PYTHON conf var to allow global override of used python + interpreter. + + Enforced usage of "python2" in PATH as default interpreter for + internal use _and_ to run duplicity (setup.py changed the + shebang to the fixed path /usr/bin/python until 0.7.05, which + we circumvent this way). + + Feature 36: Support gpg-connect-agent as a means to detect if + an agent is running, used gpg-agent for detection though. + + Quotewrapped run_cmd parameters to protect it from spaces eg. + in TMP path. + + Key export routine respects gpg-agent usage now. +- Update copyright year. +- Add ChangeLog to package. + +------------------------------------------------------------------- Old: ---- duply_1.10.1.tgz New: ---- duply_1.11.3.tgz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ duply.spec ++++++ --- /var/tmp/diff_new_pack.Hpz7kC/_old 2016-09-30 15:33:17.000000000 +0200 +++ /var/tmp/diff_new_pack.Hpz7kC/_new 2016-09-30 15:33:17.000000000 +0200 @@ -1,8 +1,8 @@ # # spec file for package duply # -# Copyright (c) 2015 SUSE LINUX Products GmbH, Nuernberg, Germany. -# Copyright (c) 2011-2015 Malcolm J Lewis <[email protected]> +# Copyright (c) 2016 SUSE LINUX GmbH, Nuernberg, Germany. +# Copyright (c) 2011-2016 Malcolm J Lewis <[email protected]> # # All modifications and additions to the file contributed by third parties # remain the property of their copyright owners, unless otherwise agreed @@ -18,13 +18,13 @@ Name: duply -Version: 1.10.1 +Version: 1.11.3 Release: 0 Summary: A frontend for the mighty duplicity magic License: GPL-2.0 Group: Productivity/Archiving/Compression Url: http://duply.net/ -Source0: http://downloads.sourceforge.net/ftplicity/duply%20%28simple%20duplicity%29/1.10.x/%{name}_%{version}.tgz +Source0: http://downloads.sourceforge.net/ftplicity/duply%20%28simple%20duplicity%29/1.11.x/%{name}_%{version}.tgz # MANUAL BEGIN Requires: duplicity # MANUAL END @@ -47,7 +47,7 @@ %files %defattr(-,root,root,-) -%doc gpl-2.0.txt +%doc CHANGELOG.txt gpl-2.0.txt %{_bindir}/%{name} %changelog ++++++ duply_1.10.1.tgz -> duply_1.11.3.tgz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/duply_1.10.1/CHANGELOG.txt new/duply_1.11.3/CHANGELOG.txt --- old/duply_1.10.1/CHANGELOG.txt 1970-01-01 01:00:00.000000000 +0100 +++ new/duply_1.11.3/CHANGELOG.txt 2016-05-29 13:16:04.000000000 +0200 @@ -0,0 +1,394 @@ +############################################################################### +TODO/IDEAS/KNOWN PROBLEMS: +- possibility to restore time frames (incl. deleted files) + realizable by listing each backup and restore from + oldest to the newest, problem: not performant +- search file in all backups function and show available + versions with backups date (list old avail since 0.6.06) +- edit profile opens conf file in vi +- implement log-fd interpretation +- add a duplicity option check against the options pending + deprecation since 0.5.10 namely --time-separator + --short-filenames + --old-filenames +- add 'exclude_<command>' list usage eg. exclude_verify +- featreq 25: a download/install duplicity option +- hint on install software if a piece is missing +- import/export profile from/to .tgz function !!! + +CHANGELOG: +1.11.3 (29.5.2016) +- fix wrong "WARNING: No running gpg-agent ..." when sign key was not set + +1.11.2 (11.2.2016) +- fix "gpg: unsafe" version print out +- bugfix 91: v1.11 [r47] broke asymmetric encryption when using GPG_KEYS_ENC +- bugfix 90: S3: TARGET_USER/PASS have no effect, added additional + documentation about needed env vars to template conf file + +1.11.1 (18.12.2015) +- bugfix 89: "Duply has trouble with PYTHON-interpreter" on OSX homebrew +- reverted duply's default PYTHON to 'python' + +1.11 (24.11.2015) +- remove obsolete --ssh-askpass routine +- add PYTHON conf var to allow global override of used python interpreter +- enforced usage of "python2" in PATH as default interpreter for internal + use _and_ to run duplicity (setup.py changed the shebang to the fixed + path /usr/bin/python until 0.7.05, which we circumvent this way) +- featreq 36: support gpg-connect-agent as a means to detect if an agent is + running (thx Thomas Harning Jr.), used gpg-agent for detection though +- quotewrapped run_cmd parameters to protect it from spaces eg. in TMP path +- key export routine respects gpg-agent usage now + +1.10.1 (19.8.2015) +- bugfix 86: Duply+Swift outputs warning +- bugfix 87: Swift fails without BACKEND_URL + +1.10 (31.7.2015) +- featreq 37: busybox issues - fix awk, grep version detection, + fix grep failure because --color=never switch is unsupported + (thx Thomas Harning Jr. for reporting and helping to debug/fix it) +- bugfix 81: --exclude-globbing-filelist is deprecated since 0.7.03 + (thx Joachim Wiedorn, also for maintaining the debian package) +- implemented base-/dirname as bash functions +- featreq 31 " Support for duplicity Azure backend " - ignored a + contributed patch by Scott McKenzie and instead opted for removing almost + all code that deals with special env vars required by backends. + adding and modifying these results in too much overhead so i dropped this + feature. the future alternative for users is to consult the duplicity + manpage and add the needed export definitions to the conf file. + appended a commented example to the template conf below the auth section. + +1.9.2 (21.6.2015) +- bugfix: exporting keys with gpg2.1 works now (thx Philip Jocks) +- documented GPG_OPTS needed for gpg2.1 to conf template (thx Troy Engel) +- bugfix 82: GREP_OPTIONS=--color=always disrupted time calculation +- added GPG conf var (see conf template for details) +- added grep version output as it is an integral needed binary +- added PYTHONPATH printout in version output + +1.9.1 (13.10.2014) +- export CMD_ERR now for scripts to detect if CMD_PREV failed/succeeded +- bugfix: CMD_PREV contained command even if it was skipped + +1.9.0 (24.8.2014) +- bugfix: env vars were not exported when external script was executable +- rework GPG_KEY handling, allow virtually anything now (uid, keyid etc.) + see gpg manpage, section "How to specify a user ID" + let gpg complain when the delivered values are invalid for whatever reason +- started to rework tmp space checking, exposed folder & writable check + TODO: reimplement enough file space available checking + +1.8.0 (13.7.2014) +- add command verifyPath to expose 'verify --file-to-restore' action +- add time parameter support to verify command +- add section time formats to usage output + +1.7.4 (24.6.2014) +- remove ubuntu one support, service is discontinued +- featreq 31: add authenticated swift (contributed by Justus Seifert) + +1.7.3 (3.4.2014) +- bugfix: test routines, gpg2 asked for passphrase although GPG_PW was set + +1.7.2 (1.4.2014 "April,April") +- bugfix: debian Bug#743190 "duply no longer allows restoration without + gpg passphrase in conf file" + GPG_AGENT_INFO env var is now needed to trigger --use-agent +- bugfix: gpg keyenc test routines didn't work if GPG_PW was not set + +1.7.1 (30.3.2014) +- bugfix: purge-* commands renamed to purgeFull, purgeIncr due to + incompatibility with new minus batch separator + +1.7.0 (20.3.2014) +- disabled gpg key id plausibility check, too many valid possibilities +- featreq 7 "Halt if precondition fails": + added and(+), or(-) batch command(separator) support +- featreq 26 "pre/post script with shebang line": + if a script is flagged executable it's executed in a subshell + now as opposed to sourced to bash, which is the default +- bugfix: do not check if dpbx, swift credentials are set anymore +- bugfix: properly escape profile name, archdir if used as arguments +- add DUPL_PRECMD conf setting for use with e.g. trickle + +1.6.0 (1.1.2014) +- support gs backend +- support dropbox backend +- add gpg-agent support to gpg test routines +- autoenable --use-agent if passwords were not defined in config +- GPG_OPTS are now honored everywhere, keyrings or complete gpg + homedir can thus be configured to be located anywhere +- always import both secret and public key if avail from config profile +- new explanatory comments in initial exclude file +- bugfix 7: Duply only imports one key at a time + +1.5.11 (19.07.2013) +- purge-incr command for remove-all-inc-of-but-n-full feature added + patch provided by Moritz Augsburger, thanks! +- documented version command in man page + +1.5.10 (26.03.2013) +- minor indent and documentation fixes +- bugfix: exclude filter failed on ubuntu, mawk w/o posix char class support +- bugfix: fix url_decoding generally and for python3 +- bugfix 3609075: wrong script results in status line (thx David Epping) + +1.5.9 (22.11.2012) +- bugfix 3588926: filter --exclude* params for restore/fetch ate too much +- restore/fetch now also ignores --include* or --exclude='foobar' + +1.5.8 (26.10.2012) +- bugfix 3575487: implement proper cloud files support + +1.5.7 (10.06.2012) +- bugfix 3531450: Cannot use space in target URL (file:///) anymore + +1.5.6 (24.5.2012) +- commands purge, purge-full have no default value anymore for security + reasons; instead max value can be given via cmd line or must be set + in profile; else an error is shown. +- minor man page modifications + +versioning scheme will be simplified to [major].[minor].[patch] version +with the next version raise + +1.5.5.5 (4.2.2012) +- bugfix 3479605: SEL context confused profile folder's permission check +- colon ':' in url passphrase got ignored, added python driven url_decoding + for user & pass to better process special chars + +1.5.5.4 (16.10.2011) +- bugfix 3421268: SFTP passwords from conf ignored and always prompted for +- add support for separate sign passphrase (needs duplicity 0.6.14+) + +1.5.5.3 (1.10.2011) +- bugfix 3416690: preview threw echo1 error +- fix unknown cmds error usage & friends if more than 2 params were given + +1.5.5.2 (23.9.2011) +- bugfix 3409643: ssh key auth did ask for passphrase (--ssh-askpass ?) +- bugfix: mawk does not support \W and did not split multikey definitions +- all parameters should survive single (') and double (") quotes now + +1.5.5.1 (7.6.2011) +- featreq 3311881: add ftps as supported by duplicity 0.6.13 (thx mape2k) +- bugfix 3312208: signing detection broke symmetric gpg test routine + +1.5.5 (2.5.2011) +- bugfix: fetch problem with space char in path, escape all params + containing non word chars +- list available profiles, if given profile cannot be found +- added --use-agent configuration hint +- bugfix 3174133: --exclude* params in conf DUPL_PARAMS broke + fetch/restore +- version command now prints out 'using installed' info +- featreq 3166169: autotrust imported keys, based on code submitted by + Martin Ellis - imported keys are now automagically trusted ultimately +- new txt2man feature to create manpages for package maintainers + +1.5.4.2 (6.1.2011) +- new command changelog +- bugfix 3109884: freebsd awk segfaulted on printf '%*', use print again +- bugfix: freebsd awk hangs on 'awk -W version' +- bugfix 3150244: mawk does not know '--version' +- minor help text improvements +- new env vars CMD_PREV,CMD_NEXT replacing CMD env var for scripts + +1.5.4.1 (4.12.2010) +- output awk, python, bash version now in prolog +- shebang uses /usr/bin/env now for freebsd compatibility, + bash not in /bin/bash +- new --disable-encryption parameter, + to override profile encr settings for one run +- added exclude-if-present setting to conf template +- bug 3126972: GPG_PW only needed for signing/symmetric encryption + (even though duplicity still needs it) + +1.5.4 (15.11.2010) +- as of 1.5.3 already, new ARCH_DIR config option +- multiple key support +- ftplicity-Feature Requests-2994929: separate encryption and signing key +- key signing of symmetric encryption possible (duplicity patch committed) +- gpg tests disable switch +- gpg tests now previewable and more intelligent + +1.5.3 (1.11.2010) +- bugfix 3056628: improve busybox compatibility, grep did not have -m param +- bugfix 2995408: allow empty password for PGP key +- bugfix 2996459: Duply erroneously escapes '-' symbol in username +- url_encode function is now pythonized +- rsync uses FTP_PASSWORD now if duplicity 0.6.10+ , else issue warning +- feature 3059262: Make pre and post aware of parameters, + internal parameters + CMD of pre or post + +1.5.2.3 (16.4.2010) +- bugfix: date again, should now work virtually anywhere + +1.5.2.2 (3.4.2010) +- minor bugfix: duplicity 0.6.8b version string now parsable +- added INSTALL.txt + +1.5.2.1 (23.3.2010) +- bugfix: date formatting is awked now and should work on all platforms + +1.5.2 (2.3.2010) +- bugfix: errors print to STD_ERR now, failed tasks print an error message +- added --name=duply_<profile> for duplicity 0.6.01+ to name cache folder +- simplified & cleaned profileless commands, removed second instance +- generalized separator time routines +- added support for --no-encryption (GPG_KEY='disabled'), see conf examples +- minor fixes + +1.5.1.5 (5.2.2010) +- bugfix: added special handling of credentials for rsync, imap(s) + +1.5.1.4 (7.1.2010) +- bugfix: nsecs defaults now to zeroes if date does not deliver [0-9]{9} +- check if ncftp binary is available if url protocol is ftp +- bugfix: duplicity output is now printed to screen directly to resolve + 'mem alloc problem' bug report +- bugfix: passwords will not be in the url anymore to solve the 'duply shows + sensitive data in process listing' bug report + +1.5.1.3 (24.12.2009) 'merry xmas' +- bugfix: gpg pass now apostrophed to allow space and friends +- bugfix: credentials are now url encoded to allow special chars in them + a note about url encoding has been added to the conf template + +1.5.1.2 (1.11.2009) +- bugfix: open parenthesis in password broke duplicity execution +- bugfix: ssh/scp backend does not always need credentials e.g. key auth + +1.5.1.1 (21.09.2009) +- bugfix: fixed s3[+http] TARGET_PASS not needed routine +- bugfix: TYPO in duply 1.5.1 prohibited the use of /etc/duply + see https://sourceforge.net/tracker/index.php?func=detail&; + aid=2864410&group_id=217745&atid=1041147 + +1.5.1 (21.09.2009) - duply (fka. ftplicity) +- first things first: ftplicity (being able to support all backends since + some time) will be called duply (fka. ftplicity) from now on. The addendum + is for the time being to circumvent confusion. +- bugfix: exit code is 1 (error) not 0 (success), if at least on duplicity + command failed +- s3[+http] now supported natively by translating user/pass to access_key/ + secret_key environment variables needed by duplicity s3 boto backend +- bugfix: additional output lines do not confuse version check anymore +- list command supports now age parameter (patch by stefan on feature + request tracker) +- bugfix: option/param pairs are now correctly passed on to duplicity +- bugfix: s3[+http] needs no TARGET_PASS if command is read only + +1.5.0.2 (31.07.1009) +- bugfix: insert password in target url didn't work with debian mawk + related to previous bug report + +1.5.0.1 (23.07.2009) +- bugfix: gawk gensub dependency raised an error on debian's default mawk + replaced with match/substr command combination (bug report) + https://sf.net/tracker/?func=detail&atid=1041147&aid=2825388&; + group_id=217745 + +1.5.0 (01.07.2009) +- removed ftp limitation, all duplicity backends should work now +- bugfix: date for separator failed on openwrt busybox date, added a + detecting workaround, milliseconds are not available w/ busybox date + +1.4.2.1 (14.05.2009) +- bugfix: free temp space detection failed with lvm, fixed awk parse routine + +1.4.2 (22.04.2009) +- gpg keys are now exported as gpgkey.[id].asc , the suffix reflects the + armored ascii nature, the id helps if the key is switched for some reason + im/export routines are updated accordingly (import is backward compatible + to the old profile/gpgkey files) +- profile argument is treated as path if it contains slashes + (for details see usage) +- non-ftplicity options (all but --preview currently) are now passed + on to duplicity +- removed need for stat in secure_conf, it is ls based now +- added profile folder readable check +- added gpg version & home info output +- awk utility availability is now checked, because it was mandatory already +- tmp space is now checked on writability and space requirement + test fails on less than 25MB or configured $VOLSIZE, + test warns if there is less than two times $VOLSIZE because + that's required for --asynchronous-upload option +- gpg functionality is tested now before executing duplicity + test drive contains encryption, decryption, comparison, cleanup + this is meant to detect non trusted or other gpg errors early +- added possibility of doing symmetric encryption with duplicity + set GPG_KEY="" or simply comment it out +- added hints in config template on the depreciation of + --short-filenames, --time-separator duplicity options + +new versioning scheme 1.4.2b => 1.4.2, +beta b's are replaced by a patch count number e.g. 1.4.2.1 will be assigned +to the first bug fixing version and 1.4.2.2 to the second and so on +also the releases will now have a release date formatted (Day.Month.Year) + +1.4.1b1 - bugfix: ftplicity changed filesystem permission of a folder + named exactly as the profile if existing in executing dir + - improved plausibility checking of config and profile folder + - secure_conf only acts if needed and prints a warning now + +1.4.1b - introduce status (duplicity collection-status) command + - pre/post script output printed always now, not only on errors + - new config parameter GPG_OPTS to pass gpg options + added examples & comments to profile template conf + - reworked separator times, added duration display + - added --preview switch, to preview generated command lines + - disabled MAX_AGE, MAX_FULL_BACKUPS, VERBOSITY in generated + profiles because they have reasonable defaults now if not set + +1.4.0b1 - bugfix: incr forces incremental backups on duplicity, + therefore backup translates to pre_bkp_post now + - bugfix: new command bkp, which represents duplicity's + default action (incr or full if full_if_older matches + or no earlier backup chain is found) + +new versioning scheme 1.4 => 1.4.0, added new minor revision number +this is meant to slow down the rapid version growing but still keep +versions cleanly separated. +only additional features will raise the new minor revision number. +all releases start as beta, each bugfix release will raise the beta +count, usually new features arrive before a version 'ripes' to stable + +1.4.0b + 1.4b - added startup info on version, time, selected profile + - added time output to separation lines + - introduced: command purge-full implements duplicity's + remove-all-but-n-full functionality (patch by unknown), + uses config variable $MAX_FULL_BACKUPS (default = 1) + - purge config var $MAX_AGE defaults to 1M (month) now + - command full does not execute pre/post anymore + use batch command pre_full_post if needed + - introduced batch mode cmd1_cmd2_etc + (in turn removed the bvp command) + - unknown/undefined command issues a warning/error now + - bugfix: version check works with 0.4.2 and older now + 1.3b3 - introduced pre/post commands to execute/debug scripts + - introduced bvp (backup, verify, purge) + - bugfix: removed need for awk gensub, now mawk compatible + 1.3b2 - removed pre/post need executable bit set + - profiles now under ~/.ftplicity as folders + - root can keep profiles in /etc/ftplicity, folder must be + created by hand, existing profiles must be moved there + - removed ftplicity in path requirement + - bugfix: bash < v.3 did not know '=~' + - bugfix: purge works again + 1.3 - introduces multiple profiles support + - modified some script errors/docs + - reordered gpg key check import routine + - added 'gpg key id not set' check + - added error_gpg (adds how to setup gpg key howto) + - bugfix: duplicity 0.4.4RC4+ parameter syntax changed + - duplicity_version_check routine introduced + - added time separator, shortnames, volsize, full_if_older + duplicity options to config file (inspired by stevie + from http://weareroot.de) + 1.1.1 - bugfix: encryption reactivated + 1.1 - introduced config directory + 1.0 - first release diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/duply_1.10.1/duply new/duply_1.11.3/duply --- old/duply_1.10.1/duply 2015-08-19 16:57:44.000000000 +0200 +++ new/duply_1.11.3/duply 2016-05-29 13:16:04.000000000 +0200 @@ -1,20 +1,20 @@ #!/usr/bin/env bash # -############################################################################### -# duply (grown out of ftplicity), is a shell front end to duplicity that # -# simplifies the usage by managing settings for backup jobs in profiles. # -# It supports executing multiple commands in a batch mode to enable single # -# line cron entries and executes pre/post backup scripts. # -# Since version 1.5.0 all duplicity backends are supported. Hence the name # -# changed from ftplicity to duply. # -# See http://duply.net or http://ftplicity.sourceforge.net/ for more info. # -# (c) 2006 Christiane Ruetten, Heise Zeitschriften Verlag, Germany # -# (c) 2008-2015 Edgar Soldin (changes since version 1.3) # -############################################################################### -# LICENSE: # -# This program is licensed under GPLv2. # -# Please read the accompanying license information in gpl.txt. # -############################################################################### +################################################################################ +# duply (grown out of ftplicity), is a shell front end to duplicity that # +# simplifies the usage by managing settings for backup jobs in profiles. # +# It supports executing multiple commands in a batch mode to enable single # +# line cron entries and executes pre/post backup scripts. # +# Since version 1.5.0 all duplicity backends are supported. Hence the name # +# changed from ftplicity to duply. # +# See http://duply.net or http://ftplicity.sourceforge.net/ for more info. # +# (c) 2006 Christiane Ruetten, Heise Zeitschriften Verlag, Germany # +# (c) 2008-2016 Edgar Soldin (changes since version 1.3) # +################################################################################ +# LICENSE: # +# This program is licensed under GPLv2. # +# Please read the accompanying license information in gpl.txt. # +################################################################################ # TODO/IDEAS/KNOWN PROBLEMS: # - possibility to restore time frames (incl. deleted files) # realizable by listing each backup and restore from @@ -32,14 +32,37 @@ # - hint on install software if a piece is missing # - import/export profile from/to .tgz function !!! # -# # CHANGELOG: +# 1.11.3 (29.5.2016) +# - fix wrong "WARNING: No running gpg-agent ..." when sign key was not set +# +# 1.11.2 (11.2.2016) +# - fix "gpg: unsafe" version print out +# - bugfix 91: v1.11 [r47] broke asymmetric encryption when using GPG_KEYS_ENC +# - bugfix 90: S3: TARGET_USER/PASS have no effect, added additional +# documentation about needed env vars to template conf file +# +# 1.11.1 (18.12.2015) +# - bugfix 89: "Duply has trouble with PYTHON-interpreter" on OSX homebrew +# - reverted duply's default PYTHON to 'python' +# +# 1.11 (24.11.2015) +# - remove obsolete --ssh-askpass routine +# - add PYTHON conf var to allow global override of used python interpreter +# - enforced usage of "python2" in PATH as default interpreter for internal +# use _and_ to run duplicity (setup.py changed the shebang to the fixed +# path /usr/bin/python until 0.7.05, which we circumvent this way) +# - featreq 36: support gpg-connect-agent as a means to detect if an agent is +# running (thx Thomas Harning Jr.), used gpg-agent for detection though +# - quotewrapped run_cmd parameters to protect it from spaces eg. in TMP path +# - key export routine respects gpg-agent usage now +# # 1.10.1 (19.8.2015) # - bugfix 86: Duply+Swift outputs warning # - bugfix 87: Swift fails without BACKEND_URL # # 1.10 (31.7.2015) -# - featreq 36: busybox issues - fix awk, grep version detection, +# - featreq 37: busybox issues - fix awk, grep version detection, # fix grep failure because --color=never switch is unsupported # (thx Thomas Harning Jr. for reporting and helping to debug/fix it) # - bugfix 81: --exclude-globbing-filelist is deprecated since 0.7.03 @@ -385,7 +408,7 @@ # 1.1.1 - bugfix: encryption reactivated # 1.1 - introduced config directory # 1.0 - first release -############################################################################### +################################################################################ # utility functions overriding binaries @@ -404,9 +427,16 @@ echo ${1%/*} } -# a lookup function for executables working with names or file paths +# implement basic which in plain bash +function which { + type -p "$@" +} + +# check availability of executables via file name or file paths function lookup { local bin="$1" + # look for file names in path via bash hash OR + # look for executables at given relative/absolute location ( [ "${bin##*/}" == "$bin" ] && hash "$bin" 2>/dev/null ) || [ -x "$bin" ] } @@ -415,7 +445,7 @@ ME_LONG="$0" ME="$(basename $0)" ME_NAME="${ME%%.*}" -ME_VERSION="1.10.1" +ME_VERSION="1.11.3" ME_WEBSITE="http://duply.net"; # default config values @@ -426,6 +456,7 @@ DEFAULT_GPG='gpg' DEFAULT_GPG_KEY='_KEY_ID_' DEFAULT_GPG_PW='_GPG_PASSWORD_' +DEFAULT_PYTHON='python' # function definitions ########################## @@ -486,14 +517,19 @@ END } +function python_binary { + echo "${PYTHON-$DEFAULT_PYTHON}" +} + function using_info { lookup duplicity && duplicity_version_get local NOTFOUND="MISSING" # freebsd awk (--version only), debian mawk (-W version only), deliver '' so awk does not wait for input local AWK_VERSION=$( lookup awk && (awk --version 2>/dev/null || awk -W version 2>&1) | awk 'NR<=2&&tolower($0)~/(busybox|awk)/{success=1;print;exit} END{if(success<1) print "unknown"}' || echo "$NOTFOUND" ) local GREP_VERSION=$( lookup grep && grep --version 2>&1 | awk 'NR<=2&&tolower($0)~/(busybox|grep.*[0-9]+\.[0-9]+)/{success=1;print;exit} END{if(success<1) print "unknown"}' || echo "$NOTFOUND" ) - local PYTHON_VERSION=$(lookup python && python -V 2>&1| awk '{print tolower($0);exit}' || echo "python $NOTFOUND" ) - local GPG_INFO=$(gpg_avail && gpg --version 2>&1| awk 'NR==1{v=$1" "$3};/^Home:/{print v" ("$0")"}' || echo "gpg $NOTFOUND") + local PYTHON_RUNNER=$(python_binary) + local PYTHON_VERSION=$(lookup "$PYTHON_RUNNER" && "$PYTHON_RUNNER" -V 2>&1| awk '{print tolower($0);exit}' || echo "'$PYTHON_RUNNER' $NOTFOUND" ) + local GPG_INFO=$(gpg_avail && gpg --version 2>&1| awk '/^gpg.*[0-9\.]+$/&&length(v)<1{v=$1" "$3}/^Home:/{h=" ("$0")"}END{print v""h}' || echo "gpg $NOTFOUND") local BASH_VERSION=$(bash --version | awk 'NR==1{IGNORECASE=1;sub(/GNU bash, version[ ]+/,"",$0);print $0}') echo -e "Using installed duplicity version ${DUPL_VERSION:-$NOTFOUND}\ ${PYTHON_VERSION+, $PYTHON_VERSION${PYTHONPATH:+ 'PYTHONPATH=$PYTHONPATH'}}\ @@ -512,8 +548,9 @@ It simplifies running duplicity with cron or on command line by: - keeping recurring settings in profiles per backup job - - enabling batch operations eg. backup_verify_purge - - executing pre/post scripts for every command + - enabling batch operations eg. backup_verify+purge + - executing pre/post scripts (different actions possible + depending on previous or next command or it's exit status) - precondition checking for flawless duplicity operation For each backup job one configuration profile must be created. @@ -748,9 +785,9 @@ # sftp://bob:[email protected]//home/bob/dupbkp # for details and available backends see duplicity manpage, section URL Format # http://duplicity.nongnu.org/duplicity.1.html#sect7 -# NOTE: -# some backends (eg. cloudfiles) need additional env vars to be set to -# work properly, when in doubt consult the man page mentioned above. +# BE AWARE: +# some backends (cloudfiles, S3 etc.) need additional env vars to be set to +# work properly, read after the TARGET definition for more details. # ATTENTION: # characters other than A-Za-z0-9.-_.~ in the URL have to be # replaced by their url encoded pendants, see @@ -760,15 +797,25 @@ TARGET='${DEFAULT_TARGET}' # optionally the username/password can be defined as extra variables # setting them here _and_ in TARGET results in an error +# ATTENTION: +# there are backends that do not support the user/pass auth scheme. +# prominent examples are S3, Azure, Cloudfiles. when in doubt consult the +# duplicity manpage. usually there is a NOTE section explaining if and which +# env vars should be set. #TARGET_USER='${DEFAULT_TARGET_USER}' #TARGET_PASS='${DEFAULT_TARGET_PASS}' -# alternatively you might export the auth env vars for your backend here -# when in doubt consult (if existing) the NOTE section of your backend on -# http://duplicity.nongnu.org/duplicity.1.html for details # eg. for cloud files backend it might look like this (uncomment for use!) #export CLOUDFILES_USERNAME='someuser' #export CLOUDFILES_APIKEY='somekey' #export CLOUDFILES_AUTHURL ='someurl' +# the following is an incomplete list (<backend>: comma separated env vars list) +# Azure: AZURE_ACCOUNT_NAME, AZURE_ACCOUNT_KEY +# Cloudfiles: CLOUDFILES_USERNAME, CLOUDFILES_APIKEY, CLOUDFILES_AUTHURL +# Google Cloud Storage: GS_ACCESS_KEY_ID, GS_SECRET_ACCESS_KEY +# Pydrive: GOOGLE_DRIVE_ACCOUNT_KEY, GOOGLE_DRIVE_SETTINGS +# S3: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY +# Swift: SWIFT_USERNAME, SWIFT_PASSWORD, SWIFT_AUTHURL, +# SWIFT_TENANTNAME OR SWIFT_PREAUTHURL, SWIFT_PREAUTHTOKEN # base directory to backup SOURCE='${DEFAULT_SOURCE}' @@ -778,6 +825,10 @@ # "trickle -s -u 640 -d 5120" # 5Mb up, 40Mb down" #DUPL_PRECMD="" +# override the used python interpreter, defaults to "python" +# e.g. "python2" or "/usr/bin/python2.7" +#PYTHON="python" + # exclude folders containing exclusion file (since duplicity 0.5.14) # Uncomment the following two lines to enable this setting. #FILENAME='.duplicity-ignore' @@ -1049,12 +1100,13 @@ elif [ -n "$CMD_DISABLED" ]; then CMD_MSG="$CMD_MSG (DISABLED) - $CMD_DISABLED" else + echo -n -e "$CMD_MSG" CMD_OUT=` eval "$@" 2>&1 ` CMD_ERR=$? if [ "$CMD_ERR" = "0" ]; then - CMD_MSG="$CMD_MSG (OK)" + CMD_MSG=" (OK)" else - CMD_MSG="$CMD_MSG (FAILED)" + CMD_MSG=" (FAILED)" fi fi echo -e "$CMD_MSG" @@ -1147,9 +1199,18 @@ # init global duplicity parameters same for all tasks duplicity_params_global - var_isset 'PREVIEW' && local RUN=echo || local RUN=eval + local RUN=eval BIN=duplicity DUPL_BIN + # run in cmd line preview mode if requested + var_isset 'PREVIEW' && RUN=echo + # try to resolve duplicity path for usage with python interpreter + DUPL_BIN=$(which "$BIN") || DUPL_BIN="$BIN" + # only run with a user specific python if configured (running by default + # breaks homebrew as they place a shell wrapper for duplicity in path) + [ -n "$PYTHON" ] && [ "$PYTHON" != "$DEFAULT_PYTHON" ] &&\ + BIN="$(qw "$(python_binary)") $(qw "$DUPL_BIN")" + $RUN ${DUPL_VARS_GLOBAL} ${BACKEND_PARAMS} \ - ${DUPL_PRECMD} duplicity $DUPL_CMD $DUPL_PARAMS_GLOBAL $(duplicity_params_conf)\ +${DUPL_PRECMD} $BIN $DUPL_CMD $DUPL_PARAMS_GLOBAL $(duplicity_params_conf)\ $GPG_USEAGENT $(gpg_custom_binary) $DUPL_CMD_PARAMS ${PREVIEW:+} local ERR=$? @@ -1181,7 +1242,7 @@ echo $date && return ## some date commands do not support giving a time w/o setting it systemwide (irix,solaris,others?) # python fallback - date=$(python -c "import time;print time.strftime('${1:-$DEFAULTFORMAT}',time.localtime(${2}))" 2> /dev/null) && \ + date=$("$(python_binary)" -c "import time;print time.strftime('${1:-$DEFAULTFORMAT}',time.localtime(${2}))" 2> /dev/null) && \ echo $date && return # awk fallback date=$(awk "BEGIN{print strftime(\"${1:-$DEFAULTFORMAT}\"${2:+,$2})}" 2> /dev/null) && \ @@ -1227,7 +1288,7 @@ function url_encode { # utilize python, silently do nothing on error - because no python no duplicity - OUT=$(python -c " + OUT=$("$(python_binary)" -c " try: import urllib.request as urllib except ImportError: import urllib print(urllib.${2}quote('$1')); @@ -1316,7 +1377,7 @@ FOUND=1 CMD_MSG="Import keyfile '$FILE' to keyring" - run_cmd gpg $GPG_OPTS --batch --import "$FILE" + run_cmd gpg $GPG_OPTS --batch --import $(qw "$FILE") if [ "$?" != "0" ]; then warning "Import failed.${CMD_OUT:+\n$CMD_OUT}" ERR=1 @@ -1343,7 +1404,7 @@ with the command \"trust\" to \"ultimate\" (5) now. Exit the edit mode of gpg with \"quit\"." CMD_MSG="Running gpg to manually edit key '$KEY_ID'" - run_cmd sleep 5\; gpg $GPG_OPTS --edit-key "$KEY_ID" + run_cmd sleep 5\; gpg $GPG_OPTS --edit-key $(qw "$KEY_ID") return $ERR } @@ -1364,15 +1425,15 @@ FILE="$(gpg_keyfile "$KEY_ID" $KEY_TYPE)" if [ ! -f "$FILE" ] && eval gpg_$(tolower $KEY_TYPE)_avail \"$KEY_ID\"; then # exporting - CMD_MSG="Export $KEY_TYPE key '$KEY_ID'" + CMD_MSG="Backup $KEY_TYPE key '$KEY_ID' to profile." # gpg2.1 insists on passphrase here, gpg2.0- happily exports w/o it # we pipe an empty string when GPG_PW is not set to avoid gpg silently waiting for input - run_cmd echo $(qw $GPG_PW) \| gpg $GPG_OPTS --passphrase-fd 0 --armor --export"$(test "SEC" = "$KEY_TYPE" && echo -secret-keys)"" $(qw $KEY_ID) >> \"$TMPFILE\"" + run_cmd $(gpg_pass_pipein GPG_PW_SIGN GPG_PW) gpg $GPG_OPTS $GPG_USEAGENT $(gpg_param_passwd GPG_PW_SIGN GPG_PW) --armor --export"$(test "SEC" = "$KEY_TYPE" && echo -secret-keys)" $(qw "$KEY_ID") '>>' $(qw "$TMPFILE") CMD_ERR=$? if [ "$CMD_ERR" = "0" ]; then CMD_MSG="Write file '"$(basename "$FILE")"'" - run_cmd " mv \"$TMPFILE\" \"$FILE\"" + run_cmd mv $(qw "$TMPFILE") $(qw "$FILE") fi if [ "$CMD_ERR" != "0" ]; then @@ -1382,7 +1443,7 @@ fi # cleanup - rm "$TMPFILE" 1>/dev/null 2>&1 + rm $(qw "$TMPFILE") 1>/dev/null 2>&1 fi done done @@ -1432,13 +1493,8 @@ echo $1 | grep -q '^[0-9a-fA-F]\{8\}$' } -#function gpg_split_keyset { -# return -# awk "BEGIN{ keys=toupper(\"$@\"); gsub(/[^A-Z0-9]/,\" \",keys); print keys }" -#} - # splits a comma separated line into lines, respects escaped commas -function gpg_split_keyset2 { +function gpg_split_keyset { local LIST LIST=$(echo "$@" | awk '{ gsub(/,/,"\n",$0); gsub(/\\\n/,",",$0); print $0 }') echo -e "$LIST" @@ -1503,13 +1559,18 @@ # 0 on success # 1 if GPG_AGENT_INFO is not set # 2 if GPG_AGENT_INFO is stale +# 3 cannot connect to gpg-agent function gpg_agent_avail { local ERR=1 if var_isset GPG_AGENT_INFO; then - ps -p $(echo $GPG_AGENT_INFO|awk -F: '{print $2}') > /dev/null 2>&1 &&\ - ERR=0 || ERR=2 + ps -p $(echo $GPG_AGENT_INFO|awk -F: '{print $2}') > /dev/null 2>&1 && \ + ERR=0 || ERR=2 + else + # GPG_AGENT_INFO is deprecated in gpg2.1, + # so we try to connect to a possibly running agent here + gpg-agent > /dev/null 2>&1 && ERR=0 || ERR=3 fi - + return $ERR } @@ -1525,12 +1586,12 @@ } function gpg_avail { - lookup $(gpg_binary) + lookup "$(gpg_binary)" } # enforce the use our selected gpg binary function gpg { - command $(gpg_binary) "$@" + command "$(gpg_binary)" "$@" } export -f gpg @@ -1621,7 +1682,7 @@ duplicity_version_check # check for certain important helper programs -for f in awk grep; do +for f in awk grep "$(python_binary)"; do lookup "$f" || \ error_path "$f missing. installed und available in path?" done @@ -1784,10 +1845,13 @@ # GPG create key settings, config check2 (needs gpg) ########################## if gpg_disabled; then - : # the following tests are not necessary + : # the following tests are not necessary else -# key set? +# we test this early as any invocation gpg2.1+ starts gpg-agent automatically +GPG_AGENT_ERR=$(gpg_agent_avail ; echo $?) + +# enc key still default? if [ "$GPG_KEY" == "${DEFAULT_GPG_KEY}" ]; then error_gpg "Encryption Key GPG_KEY still default in conf file '$CONF'." @@ -1795,9 +1859,30 @@ # create array of gpg encr keys, for further processing OIFS="$IFS" IFS=$'\n' -GPG_KEYS_ENC_ARRAY=( $( gpg_split_keyset2 ${GPG_KEY},${GPG_KEYS_ENC} ) ) +GPG_KEYS_ENC_ARRAY=( $( gpg_split_keyset ${GPG_KEY},${GPG_KEYS_ENC} ) ) IFS="$OIFS" +# pw set? +# symmetric needs one, always +if gpg_symmetric && ( [ -z "$GPG_PW" ] || [ "$GPG_PW" == "${DEFAULT_GPG_PW}" ] ) \ + ; then + error_gpg "Encryption passphrase GPG_PW (needed for symmetric encryption) +is empty/not set or still default value in conf file +'$CONF'." +fi +# this is a technicality, we can only pump one pass via pipe into gpg +# but symmetric already always needs one for encryption +if gpg_symmetric && var_isset GPG_PW && var_isset GPG_PW_SIGN &&\ + [ -n "$GPG_PW_SIGN" ] && [ "$GPG_PW" != "$GPG_PW_SIGN" ]; then + error_gpg "GPG_PW _and_ GPG_PW_SIGN are defined but not identical in config +'$CONF'. +This is unfortunately impossible. For details see duplicity manpage, +section 'A Note On Symmetric Encryption And Signing'. + +Tip: Separate signing keys may have empty passwords e.g. GPG_PW_SIGN=''. +Tip2: Use gpg-agent." +fi + # check gpg encr public keys availability for (( i = 0 ; i < ${#GPG_KEYS_ENC_ARRAY[@]} ; i++ )); do KEY_ID="${GPG_KEYS_ENC_ARRAY[$i]}" @@ -1850,26 +1935,7 @@ fi fi -# pw set? -# symmetric needs one, always -if gpg_symmetric && ( [ -z "$GPG_PW" ] || [ "$GPG_PW" == "${DEFAULT_GPG_PW}" ] ) \ - ; then - error_gpg "Encryption passphrase GPG_PW (needed for symmetric encryption) -is empty/not set or still default value in conf file -'$CONF'." -fi -# this is a technicality, we can only pump one pass via pipe into gpg -# but symmetric already always needs one for encryption -if gpg_symmetric && var_isset GPG_PW && var_isset GPG_PW_SIGN &&\ - [ -n "$GPG_PW_SIGN" ] && [ "$GPG_PW" != "$GPG_PW_SIGN" ]; then - error_gpg "GPG_PW _and_ GPG_PW_SIGN are defined but not identical in config -'$CONF'. -This is unfortunately impossible. For details see duplicity manpage, -section 'A Note On Symmetric Encryption And Signing'. - -Tip: Separate signing keys may have empty passwords e.g. GPG_PW_SIGN=''. -Tip2: Use gpg-agent." -fi +# using GPG_AGENT_ERR set early above, try to autoenable gpg-agent or issue some warnings # key enc can deal without, but might profit from gpg-agent # if GPG_PW is not set alltogether # if signing key is different from first (main) enc key (we can only pipe one pass into gpg) @@ -1877,13 +1943,14 @@ ( ! var_isset GPG_PW || \ ( gpg_signing && ! var_isset GPG_PW_SIGN && [ "$GPG_KEY_SIGN" != "${GPG_KEYS_ENC_ARRAY[0]}" ] ) ); then - GPG_AGENT_ERR=$(gpg_agent_avail ; echo $?) if [ "$GPG_AGENT_ERR" -eq 1 ]; then - echo "Cannot use gpg-agent. GPG_AGENT_INFO not set." + warning "Cannot use gpg-agent. GPG_AGENT_INFO not set." elif [ "$GPG_AGENT_ERR" -eq 2 ]; then - echo "Cannot use gpg-agent! GPG_AGENT_INFO contains stale pid." + warning "Cannot use gpg-agent! GPG_AGENT_INFO contains stale pid." + elif [ "$GPG_AGENT_ERR" -eq 3 ]; then + warning "No running gpg-agent found although GPG_PW or GPG_PW_SIGN (enc != sign key) not set." else - echo "Autoenable use of gpg-agent. GPG_PW or GPG_PW_SIGN (enc != sign key) not set." + echo "Enable gpg-agent usage. Running gpg-agent instance found and GPG_PW or GPG_PW_SIGN (enc != sign key) not set." GPG_USEAGENT="--use-agent" fi fi @@ -1893,17 +1960,11 @@ # config plausibility check - SPACE ########################################### -# is tmp is a folder -CMD_MSG="Checking TEMP_DIR '${TEMP_DIR}' is a folder" -run_cmd test -d "$TEMP_DIR" -if [ "$?" != "0" ]; then - error "Temporary file space '$TEMP_DIR' is not a directory." -fi -# is tmp writeable -CMD_MSG="Checking TEMP_DIR '${TEMP_DIR}' is writable" -run_cmd test -w "$TEMP_DIR" +# is tmp is a folder and writable +CMD_MSG="Checking TEMP_DIR '${TEMP_DIR}' is a folder and writable" +run_cmd test -d $(qw "$TEMP_DIR") '&&' test -w $(qw "$TEMP_DIR") if [ "$?" != "0" ]; then - error "Temporary file space '$TEMP_DIR' not writable." + error "Temporary file space '$TEMP_DIR' is not a directory or writable." fi @@ -1928,7 +1989,7 @@ GPG_TEST="$TEMP_DIR/${ME_NAME}.$$.$(date_fix %s)" function cleanup_gpgtest { echo -en "Cleanup - Delete '${GPG_TEST}_*'" - rm ${GPG_TEST}_* 2>/dev/null && echo "(OK)" || echo "(FAILED)" + rm "${GPG_TEST}"_* 2>/dev/null && echo "(OK)" || echo "(FAILED)" } # signing enabled? @@ -1945,7 +2006,7 @@ done # check encrypting CMD_MSG="Test - Encrypt to '$(join "','" "${GPG_KEYS_ENC_ARRAY[@]}")'${CMD_MSG_SIGN:+ & $CMD_MSG_SIGN}" - run_cmd $(gpg_pass_pipein GPG_PW_SIGN GPG_PW) gpg $CMD_PARAM_SIGN $(gpg_param_passwd GPG_PW_SIGN GPG_PW) $CMD_PARAMS $GPG_USEAGENT --status-fd 1 $GPG_OPTS -o "${GPG_TEST}_ENC" -e "$ME_LONG" + run_cmd $(gpg_pass_pipein GPG_PW_SIGN GPG_PW) gpg $CMD_PARAM_SIGN $(gpg_param_passwd GPG_PW_SIGN GPG_PW) $CMD_PARAMS $GPG_USEAGENT --status-fd 1 $GPG_OPTS -o $(qw "${GPG_TEST}_ENC") -e $(qw "$ME_LONG") CMD_ERR=$? if [ "$CMD_ERR" != "0" ]; then @@ -1959,7 +2020,7 @@ # check decrypting CMD_MSG="Test - Decrypt" gpg_key_decryptable || CMD_DISABLED="No matching secret key available." - run_cmd $(gpg_pass_pipein GPG_PW) gpg $(gpg_param_passwd GPG_PW) $GPG_OPTS -o "${GPG_TEST}_DEC" $GPG_USEAGENT -d "${GPG_TEST}_ENC" + run_cmd $(gpg_pass_pipein GPG_PW) gpg $(gpg_param_passwd GPG_PW) $GPG_OPTS -o $(qw "${GPG_TEST}_DEC") $GPG_USEAGENT -d $(qw "${GPG_TEST}_ENC") CMD_ERR=$? if [ "$CMD_ERR" != "0" ]; then @@ -1970,7 +2031,7 @@ else # check encrypting CMD_MSG="Test - Encryption with passphrase${CMD_MSG_SIGN:+ & $CMD_MSG_SIGN}" - run_cmd $(gpg_pass_pipein GPG_PW) gpg $GPG_OPTS $CMD_PARAM_SIGN --passphrase-fd 0 -o "${GPG_TEST}_ENC" --batch -c "$ME_LONG" + run_cmd $(gpg_pass_pipein GPG_PW) gpg $GPG_OPTS $CMD_PARAM_SIGN --passphrase-fd 0 -o $(qw "${GPG_TEST}_ENC") --batch -c $(qw "$ME_LONG") CMD_ERR=$? if [ "$CMD_ERR" != "0" ]; then error_gpg_test "Encryption failed.${CMD_OUT:+\n$CMD_OUT}" @@ -1978,7 +2039,7 @@ # check decrypting CMD_MSG="Test - Decryption with passphrase" - run_cmd $(gpg_pass_pipein GPG_PW) gpg $GPG_OPTS --passphrase-fd 0 -o "${GPG_TEST}_DEC" --batch -d "${GPG_TEST}_ENC" + run_cmd $(gpg_pass_pipein GPG_PW) gpg $GPG_OPTS --passphrase-fd 0 -o $(qw "${GPG_TEST}_DEC") --batch -d $(qw "${GPG_TEST}_ENC") CMD_ERR=$? if [ "$CMD_ERR" != "0" ]; then error_gpg_test "Decryption failed.${CMD_OUT:+\n$CMD_OUT}" @@ -2055,15 +2116,6 @@ var_isset 'TARGET_URL_PASS' && BACKEND_PARAMS="IMAP_PASSWORD=$(qw "${TARGET_URL_PASS}")" ;; *) - # add needed param for ssh backend - case "$(tolower "${TARGET_URL_PROT%%:*}")" in - 'ssh'|'sftp'|'scp') - # ssh backend wants to be told that there is a pass to use - var_isset 'TARGET_URL_PASS' && \ - DUPL_PARAMS="$DUPL_PARAMS --ssh-askpass" && \ - BACKEND_PARAMS="FTP_PASSWORD=$(qw "${TARGET_URL_PASS}")" - ;; - esac # rest uses FTP_PASS var var_isset 'TARGET_URL_PASS' && \ BACKEND_PARAMS="FTP_PASSWORD=$(qw "${TARGET_URL_PASS}")"
