Re: [Duplicity-team] [Merge] lp:~breunigs/duplicity/amazondrive into lp:duplicity

[edso]

Review: Needs Information

hey Stefan,

wrt.
1.
can be disregarded if the other backend will be replaced completely

2.
yepp.. overlooked that. don't mind my comment there.

3. and 4.
same as 1.

additionally..
there were concerns when adding the Dropbox backend wrt. the trademark so it 
was added as DPBXBackend and the only mention of the Dropbox name is to what it 
is intended to connect. not sure where the limitation stems from, but you might 
consider to limit mentioning Amazon. did you check the license and terms for 
using their API wrt. to this? on the other hand shortening it from the start 
might circumvent any danger at all.

thanks for contributing.. ede/duply.net
-- 
https://code.launchpad.net/~breunigs/duplicity/amazondrive/+merge/309084
Your team duplicity-team is subscribed to branch lp:duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Merge] lp:~breunigs/duplicity/amazondrive into lp:duplicity

[edso]

Review: Approve

hey Stefan,

out of interest. what is meant by

"
109 +# NOTE: Amazon requires https, which is why I am using my 
domain/setup
110 +# instead of Duplicity's. Mail me at stefan-duplic...@breunig.xyz 
once it is
111 +# available through https and I will whitelist the new URL.
112 +OAUTH_REDIRECT_URL = 'https://breunig.xyz/duplicity/copy.html'
113 +OAUTH_SCOPE = ['clouddrive:read_other', 'clouddrive:write']
"

what would be needed here on part of duplicity. https webspace? can you please 
explain in short why it is needed, what happens?

thx.. ede

-- 
https://code.launchpad.net/~breunigs/duplicity/amazondrive/+merge/309084
Your team duplicity-team is subscribed to branch lp:duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Merge] lp:~dernils/duplicity/robust-dropbox-backend into lp:duplicity

[edso]

Review: Needs Fixing

hey Nils,

looks good, but please add the new parameter to the man page /bin/duplicity.1 
as well. no use to have a parameter that is not properly documented.

thanks.. ede
-- 
https://code.launchpad.net/~dernils/duplicity/robust-dropbox-backend/+merge/311073
Your team duplicity-team is subscribed to branch lp:duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #404141]: Invalid data - SHA1 hash mismatch for file (One cause)

[edso]

Question #404141 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/404141

edso posted a new comment:
hey Matt, thanks for sharing your experience.. ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Merge] lp:~dernils/duplicity/robust-dropbox-backend into lp:duplicity

[edso]

Review: Approve


-- 
https://code.launchpad.net/~dernils/duplicity/robust-dropbox-backend/+merge/311073
Your team duplicity-team is subscribed to branch lp:duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Merge] lp:~dernils/duplicity/robust-dropbox-backend into lp:duplicity

[edso]

Review: Needs Fixing

close, but your missing the parameter eg. 

.BI "--backend-retry-delay " number

..ede
-- 
https://code.launchpad.net/~dernils/duplicity/robust-dropbox-backend/+merge/311073
Your team duplicity-team is subscribed to branch lp:duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #404018]: Forced disconnect of my internet connection during backup

[edso]

Question #404018 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/404018

Status: Open => Answered

edso proposed the following answer:
On 12.11.2016 16:37, nils wrote:
> New question #404018 on Duplicity:
> https://answers.launchpad.net/duplicity/+question/404018
> 
> Hi there,

hey Nils, long time no read ;)

> I'm facing an interesting situation and I'm sure that I'm not alone:
> I have a backup set that comprises way more data than what my internet 
> connection can handle within 24 hours (the set is about 200 GB and that 
> should take round about a week). 
> Unfortunately, my ISP disconnects my internet connection every 24 hours and 
> assigns me a new IP. And the backends that I've tried so far (SSH and 
> dropbox) cannot handle the closed socket (even though the internet 
> connectivity is back after a few seconds). 
> I tried quite a few things but in the end failed. So, I have some questions:
> 1) Does it somehow harm the quality of the backup if I would start the backup 
> process over manually (or via the bash) 20 times? I don't find it a good 
> solution to resume the backup for so often but currently I see no other 
> option. I really would appreciate your opinion on that

it shouldn't although resuming always contains a minimal risk, that
wouldn't be there otherwise. i suggest you do regular verify runs to
make sure that yur backups are in good shape.

> 2) Are there or will there be any backends that can hanlde such a
situation? In principle, it's pretty simple. The backend "only" would
have to start over authentication and reconnect completely in case of a
permanent error (at least trying this in case of a permanent error would
be very useful).

not as such. resuming is only done on when duplicity detects a that the 
conditions are right to resume on a new backup run.
however what backends can do is retry and you may finetune the retry behaviour 
via --num-retries . the delay is currently hardcoded as 30s in 
http://bazaar.launchpad.net/~duplicity-team/duplicity/0.8-series/view/head:/duplicity/backend.py#L400

> 3) Is anybody here encountering the same problem and maybe found a
different solution that I did not yet think of?

probably. the usual workaround that is mentioned on the ml for issues
like that is to backup to a local file:// target and rsync or use some
other means to sync it to the backend of your choice. this way the
backup process does not get interrupted by uplink issues.

have fun ..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #404018]: Forced disconnect of my internet connection during backup

[edso]

Question #404018 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/404018

edso proposed the following answer:
On 13.11.2016 16:17, nils wrote:
> Question #404018 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/404018
> 
> nils posted a new comment:
> 
>> Hi there,
> 
> hey Nils, long time no read ;)
> 
> Nils: Indeed, currently a huge amount of my spare time somehow ends up
> with duplicity :-)
> 
>> I'm facing an interesting situation and I'm sure that I'm not alone:
>> I have a backup set that comprises way more data than what my internet 
>> connection can handle within 24 hours (the set is about 200 GB and that 
>> should take round about a week).
>> Unfortunately, my ISP disconnects my internet connection every 24 hours and 
>> assigns me a new IP. And the backends that I've tried so far (SSH and 
>> dropbox) cannot handle the closed socket (even though the internet 
>> connectivity is back after a few seconds).
>> I tried quite a few things but in the end failed. So, I have some questions:
>> 1) Does it somehow harm the quality of the backup if I would start the 
>> backup process over manually (or via the bash) 20 times? I don't find it a 
>> good solution to resume the backup for so often but currently I see no other 
>> option. I really would appreciate your opinion on that
> 
> it shouldn't although resuming always contains a minimal risk, that
> wouldn't be there otherwise. i suggest you do regular verify runs to
> make sure that yur backups are in good shape.
> 
> Nils: OK, I guess, I'll consider going this way then.
> 
>> 2) Are there or will there be any backends that can hanlde such a
> situation? In principle, it's pretty simple. The backend "only" would
> have to start over authentication and reconnect completely in case of a
> permanent error (at least trying this in case of a permanent error would
> be very useful).
> 
> not as such. resuming is only done on when duplicity detects a that the 
> conditions are right to resume on a new backup run.
> however what backends can do is retry and you may finetune the retry 
> behaviour via --num-retries . the delay is currently hardcoded as 30s in 
> http://bazaar.launchpad.net/~duplicity-team/duplicity/0.8-series/view/head:/duplicity/backend.py#L400
> 
> Nils: I already played around with the timeouts and the numbers of
> retries but that does not help as (with the forced disconnect) the
> complete socket is gone. The backends would have to go one step further
> and repeat authentication against the backend server but they don't do
> that :-(

pretty sure that there are some that do eg. WebDAV or pexpect+ssh

if you need it, consider patching the backends or file a bug report wrt.
to this issue.

>> 3) Is anybody here encountering the same problem and maybe found a
> different solution that I did not yet think of?
> 
> probably. the usual workaround that is mentioned on the ml for issues
> like that is to backup to a local file:// target and rsync or use some
> other means to sync it to the backend of your choice. this way the
> backup process does not get interrupted by uplink issues.
> 
> Nils: Yes, I read that. For me that does not work as it would consume
> disk space that I don't have :-(
> 

makes sense.. ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

[Duplicity-team] [Merge] lp:~ed.so/duplicity/manpage.fixes into lp:duplicity

[edso]

edso has proposed merging lp:~ed.so/duplicity/manpage.fixes into lp:duplicity.

Requested reviews:
  duplicity-team (duplicity-team)

For more details, see:
https://code.launchpad.net/~ed.so/duplicity/manpage.fixes/+merge/311405

fix html output via rman on the website
-- 
Your team duplicity-team is requested to review the proposed merge of 
lp:~ed.so/duplicity/manpage.fixes into lp:duplicity.
=== modified file 'bin/duplicity.1'
--- bin/duplicity.1	2016-11-01 19:26:03 +
+++ bin/duplicity.1	2016-11-21 15:20:29 +
@@ -1726,7 +1726,6 @@
 will distinguish between different backups.
 
 .SH A NOTE ON MULTI BACKEND
-
 The multi backend allows duplicity to combine the storage available in
 more than one backend store (e.g., you can store across a google drive
 account and a onedrive account to get effectively the combined storage
@@ -1736,58 +1735,46 @@
 Each element of the list must have a "url" element, and may also contain
 an optional "description" and an optional "env" list of environment
 variables used to configure that backend.
-
 .SS Query Parameters
-
 Query parameters come after the file URL in standard HTTP format
 for example:
-
+.RS
 .nf
-.RS
 multi:///path/to/config.json?mode=mirror=abort
 multi:///path/to/config.json?mode=stripe=continue
 multi:///path/to/config.json?onfail=abort=stripe
 multi:///path/to/config.json?onfail=abort
+.fi
 .RE
-
 Order does not matter, however unrecognized parameters are considered
 an error.
-
 .TP
 .BI "mode=" stripe
-
 This mode (the default) performs round-robin access to the list of
 backends. In this mode, all backends must be reliable as a loss of one
 means a loss of one of the archive files.
-
 .TP
 .BI "mode=" mirror
-
 This mode accesses backends as a RAID1-store, storing every file in
 every backend and reading files from the first-successful backend.
 A loss of any backend should result in no failure. Note that backends
 added later will only get new files and may require a manual sync
 with one of the other operating ones.
-
 .TP
 .BI "onfail=" continue
-
 This setting (the default) continues all write operations in as
 best-effort. Any failure results in the next backend tried. Failure
 is reported only when all backends fail a given operation with the
 error result from the last failure.
-
 .TP
 .BI "onfail=" abort
-
 This setting considers any backend write failure as a terminating
 condition and reports the error.
 Data reading and listing operations are independent of this and
 will try with the next backend on failure.
-
 .SS JSON File Example
+.RS
 .nf
-.RS
 [
  {
   "description": "a comment about the backend"
@@ -1807,8 +1794,8 @@
   "url": "file:///path/to/dir"
  }
 ]
+.fi
 .RE
-.fi
 
 .SH A NOTE ON PAR2 WRAPPER BACKEND
 Par2 Wrapper Backend can be used in combination with all other backends to

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #404071]: rclone backend

[edso]

Question #404071 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/404071

Status: Open => Answered

edso proposed the following answer:
On 14.11.2016 20:23, Armin Schindler wrote:
> New question #404071 on Duplicity:
> https://answers.launchpad.net/duplicity/+question/404071
> 
> Is there any effort in having a backend for rclone yet?

never heard of it on the ml, so probably no.

> I tried acd_cli for amazon cloud drive, but it is not very reliable and slow 
> compared to rclone.
> So I wrote a simple backend for rclone. How can I contribute backend file?
> 

commit to a branch on launchpad.net.

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #404018]: Forced disconnect of my internet connection during backup

[edso]

Question #404018 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/404018

edso proposed the following answer:
On 14.11.2016 22:02, nils wrote:
> Question #404018 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/404018
> 
> nils posted a new comment:
> The patch on the Dropbox backend is working now. Will test it over the next 
> few weeks in my backup system.
> You can find it under https://github.com/henrysher/duplicity/pull/9
> 

you should drop 
  def internet_on():
and the urllib2 import, as they do not seem to be needed.

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #404018]: Forced disconnect of my internet connection during backup

[edso]

Question #404018 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/404018

edso proposed the following answer:
On 15.11.2016 14:17, nils wrote:
> Question #404018 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/404018
> 
> nils posted a new comment:
> You are right. That's a relict from my first attempt to solve the issue. I 
> just removed it. 

it's not a good idea either. the service (google web server in that
case) might be out of order for other reasons than your uplink being
down. so it's no sure way to detect anything.

the the established approach in other backends is to check the
socket/auth and recreate it as needed, as you eventually did as well.

> Regarding the configurable delay: That sounds like a very good idea. I'll 
> look into it as soon as I did some paid work ;-) I have also some loose plans 
> to update the Dropbox backend to the latest Dropbox SDK.
> 

sounds good.. ede

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #403594]: Restoring a directory from S3

[edso]

Question #403594 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/403594

Status: Open => Answered

edso proposed the following answer:
On 31.10.2016 19:32, Eric Christensen wrote:
> Question #403594 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/403594
> 
> Eric Christensen gave more information on the question:
> duplicity restore --file-to-restore "data/Eric/files/Geneology" $DEST
> /home/echristensen/Genealogy --s3-use-new-style -v8
> 
> data/Eric/files/Geneology not found in archive - no files restored.
> 
> So it doesn't appear to be a / problem.
> 

can you show the part of the output of 'duplicity list-current-files
...' that shows that the file/folder exists in the backup?

btw. is it Geneology or Genealogy?

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Merge] lp:~havard/duplicity/jottacloudbackend into lp:duplicity

[edso]

Ken, Havard,

any reason why the backend did not get documented in the man page?

how are users to know it is there and how it works?

..ede/duply.net
-- 
https://code.launchpad.net/~havard/duplicity/jottacloudbackend/+merge/304955
Your team duplicity-team is subscribed to branch lp:duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #402905]: Task 'BKP' failed with exit code '30'

[edso]

Question #402905 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/402905

edso posted a new comment:
On 01.11.2016 13:38, juniorfga wrote:
> Question #402905 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/402905
> 
> Status: Open => Solved
> 
> juniorfga confirmed that the question is solved:
> I've found the solution on this page
> 
> http://boto.cloudhackers.com/en/latest/getting_started.html
> 
> under the "Configuring Boto Credentials" topic, it seems that
> Duply/Duplicity is not updating the environment variables (e.g.
> AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SECURITY_TOKEN and
> AWS_PROFILE). so I've created a ~/.boto file with these contents:
> 
> [Credentials]
> aws_access_key_id = YOURACCESSKEY
> aws_secret_access_key = YOURSECRETKEY
> 
> and it start working
> 

Junior,

why would you expect duply/duplicity to generate the file for you? 
the boto backend sources say

"""
Backend for Amazon's Simple Storage System, (aka Amazon S3), though
the use of the boto module, (http://code.google.com/p/boto/).

To make use of this backend you must set aws_access_key_id
and aws_secret_access_key in your ~/.boto or /etc/boto.cfg
with your Amazon Web Services key id and secret respectively.
Alternatively you can export the environment variables
AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.
"""

did you export the AWS_* env vars beforehand at all?

note to self: the manpage is missing the infos above about amazon s3
access auth completely. have to update manpage accordingly.

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #402919]: single backedup file - restore problem

[edso]

Question #402919 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/402919

Status: Open => Answered

edso proposed the following answer:
On 03.11.2016 19:58, John wrote:
> Question #402919 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/402919
> 
> Status: Expired => Open
> 
> John is still having a problem:
> Unfortunately it isn't "expired" because I'm still having this problem..
> 

can you run the restore command again w/ max verbosity and redirect the
whole output (STDERR/OUT) into a file and pastebin that? check it
beforehand for infos you deem private and obfuscate those.

also please run a 
  'duplicity list-current-files ...'
and 
  'duplicity collection-status ...'
and send these outputs as well.

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #403594]: Restoring a directory from S3

[edso]

Question #403594 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/403594

Status: Open => Needs information

edso requested more information:
ok, it should be in there ;)

let's start using bigger guns now. please send/pastebin/mail me
personally (as you prefer)

1. the complete output of a restore command w/ max. verbosity '-v9'

2. the complete file listing on the backend

you may want to obfuscate private info in the output before you send it.

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Merge] lp:~horgh/duplicity/copy-symlink-targets-721599 into lp:duplicity

[edso]

On 05.12.2016 21:37, Will Storey wrote:
> Thank you, that is a lot clearer!

that's the idea ;) ..ede

-- 
https://code.launchpad.net/~horgh/duplicity/copy-symlink-targets-721599/+merge/312438
Your team duplicity-team is subscribed to branch lp:duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Merge] lp:~horgh/duplicity/copy-symlink-targets-721599 into lp:duplicity

[edso]

nP. but please change the man page text to 

+.B --copy-links
+Resolve symlinks during backup.
+Enabling this will resolve & back up the symlink's file/folder data instead of 
the symlink itself, potentially increasing the size of the backup.

just want to make sure users understand the consequences of that parameter.
btw. i am curious, why do you need it anyway?

..ede/duply.net
-- 
https://code.launchpad.net/~horgh/duplicity/copy-symlink-targets-721599/+merge/312438
Your team duplicity-team is subscribed to branch lp:duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #577611]: Duplicity never ending job, lots of instances

[edso]

Question #577611 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/577611

Status: Open => Answered

edso proposed the following answer:
On 20.03.2017 20:28, Jasper wrote:
> New question #577611 on Duplicity:
> https://answers.launchpad.net/duplicity/+question/577611
> 
> I'm running Duplicity with Duply on a couple servers with S3 as target, and 
> experience the same issue on all servers.
> 
> It seems like Duplicity never actually ends its job, resulting in constant 
> high memory usage and 38 Duplicity instances and 673 GPG instances.
> 
> duply -v
> duply version 2.0.1
> (http://duply.net)
> 
> Using installed duplicity version 0.7.11, python 2.7.6, gpg 1.4.16 (Home: 
> ~/.gnupg), awk 'GNU Awk 4.0.1', grep 'grep (GNU grep) 2.16', bash 
> '4.3.11(1)-release (x86_64-pc-linux-gnu)'.
> 
> The Duply profile is nearly identical for each server, here's an example:
> 
> GPG_KEY='FOO'
> GPG_PW='FOO'
> 
> TARGET='s3://s3-eu-central-1.amazonaws.com/foo-bucket/bar-location'
> 
> export AWS_ACCESS_KEY_ID='FOO'
> export AWS_SECRET_ACCESS_KEY='FOO'
> 
> # base directory to backup
> SOURCE='/'
> 
> # exclude folders containing exclusion file (since duplicity 0.5.14)
> # Uncomment the following two lines to enable this setting.
> FILENAME='.duplicity-ignore'
> DUPL_PARAMS="$DUPL_PARAMS --exclude-if-present '$FILENAME'"
> 
> # Time frame for old backups to keep, Used for the "purge" command.
> # see duplicity man page, chapter TIME_FORMATS)
> MAX_AGE=2M
> 
> # Number of full backups to keep. Used for the "purge-full" command.
> # See duplicity man page, action "remove-all-but-n-full".
> MAX_FULL_BACKUPS=2
> 
> # Number of full backups for which incrementals will be kept for.
> # Used for the "purge-incr" command.
> # See duplicity man page, action "remove-all-inc-of-but-n-full".
> MAX_FULLS_WITH_INCRS=1
> 
> # activates duplicity --full-if-older-than option (since duplicity v0.4.4.RC3)
> # forces a full backup if last full backup reaches a specified age, for the
> # format of MAX_FULLBKP_AGE see duplicity man page, chapter TIME_FORMATS
> # Uncomment the following two lines to enable this setting.
> MAX_FULLBKP_AGE=1M
> DUPL_PARAMS="$DUPL_PARAMS --full-if-older-than $MAX_FULLBKP_AGE "
> 
> # sets duplicity --volsize option (available since v0.4.3.RC7)
> # set the size of backup chunks to VOLSIZE MB instead of the default 25MB.
> # VOLSIZE must be number of MB's to set the volume size to.
> # Uncomment the following two lines to enable this setting.
> VOLSIZE=100
> DUPL_PARAMS="$DUPL_PARAMS --volsize $VOLSIZE "
> 
> # more duplicity command line options can be added in the following way
> # don't forget to leave a separating space char at the end
> #DUPL_PARAMS="$DUPL_PARAMS --put_your_options_here "
> 
> The backups are run via cron:
> 
> 12 3 * * * nice -n19 ionice -c2 -n7 duply database backup_verify_purge 
> --force --name foo_database >> /var/log/duplicity.log 2>&1
> 
> 26 3 * * * nice -n19 ionice -c2 -n7 duply websites backup_verify_purge 
> --force --name foo_websites >> /var/log/duplicity.log 2>&1
> 
> 53 4 * * * nice -n19 ionice -c2 -n7 duply system backup_verify_purge --force 
> --name foo_system >> /var/log/duplicity.log 2>&1
> 
> 


are you saying that multiple instances for one target are running on the same 
box?

this should be prevented by the lockfile. anyhow, if for some reason
not, that definitely leads to an undefined behaviour of several
instances writing to the same target.

so, is it possible that new instances are started while the older one is
still running?

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #601449]: How to stop ever-growing backup volume?

[edso]

Question #601449 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/601449

edso proposed the following answer:
On 03.04.2017 00:03, GilgongoJones wrote:
> Question #601449 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/601449
> 
> GilgongoJones gave more information on the question:
> I tried sftp:// and got:
> 
> BackendException: could not load ~/.ssh/known_hosts, maybe corrupt?
> 
> Then tried pexpect+sftp:// and got:
> 
> Attempt 1 failed. AttributeError: 'NoneType' object has no attribute 'encode'
> Attempt 2 failed. AttributeError: 'NoneType' object has no attribute 'encode'
> Attempt 3 failed. AttributeError: 'NoneType' object has no attribute 'encode'
> 

that sounds like a very old duplicity. please upgrade to the latest
stable 0.7.12 ..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #601449]: How to stop ever-growing backup volume?

[edso]

Question #601449 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/601449

Status: Open => Answered

edso proposed the following answer:
On 01.04.2017 08:33, GilgongoJones wrote:
> Question #601449 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/601449
> 
> Status: Answered => Open
> 
> GilgongoJones is still having a problem:
> Thanks - this is what I get with that (although I recently had to delete
> the volume, cache and run the backup from scratch).
> 
> BTW why does it also always say "Last full backup date: none"?
> 
> Local and Remote metadata are synchronized, no sync needed.
> Last full backup date: none
> Collection Status
> -
> Connecting with backend: BackendWrapper
> Archive directory: 
> /home/gilgongo/.cache/duplicity/02be6e629f6ff2159271c8f6cd84ce8d
> 
> Found 0 secondary backup chains.
> No backup chains with active signatures found
> No orphaned or incomplete backup sets found.
> 

duplicity does not seem to see any files on your backend. can you make
sure that there is at least one duplicity chain there?

using rsync backend can usually be replaced with sftp:// or
pexpect+sftp:// . maybe it is a backend issue on your side.

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #654243]: pypi package

[edso]

Question #654243 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/654243

Status: Open => Answered

edso proposed the following answer:
hi Adam,

On 31.07.2017 02:03, Adam Dobrawy wrote:
> New question #654243 on Duplicity:
> https://answers.launchpad.net/duplicity/+question/654243
> 
> 616/5000
> Is there any reason why there are no pypi packages? I created only (I have 
> only commented out SDistCommand in setup.py). I do not see problems. Pypi 
> repositories provide software availability independent of the operating 
> system. There is no need to use Ubuntu. This will allow you to use the latest 
> version of the application more widely.

probably it is just because nobody volunteered to provide them so far. if you 
know an easy way to create them on launchpad.net just tell Ken the maintainer.
 
> This would allow for frequent updates, also in compiled form (see 
> "manylinux"). I do not know the Launchpad environment, but usually it is 
> implemented by CI.
> 
> It is worth mentioning that for the "youtube-dl" program, using the pypi 
> repository is the primary form of software distribution.
> 

what is "youtube-dl" and where is it connected to duplicity?
..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #653959]: "Num Volumes" doesn't match what is expected

[edso]

Question #653959 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/653959

Status: Open => Answered

edso proposed the following answer:
answers inline

On 27.07.2017 21:03, baldwintm wrote:
> New question #653959 on Duplicity:
> https://answers.launchpad.net/duplicity/+question/653959
> 
> I am running backups using duply.
> I have "VOLSIZE = 250" in the duply conf file, backing up a directory with 
> 3.6 TB of data.
> In the output of "duplicity collection-status", I have the following backup 
> chains:
> 
> 
> Collection Status
> -
> Connecting with backend: LocalBackend
> Archive dir: 
> 
> Found 2 secondary backup chains.
> Secondary chain 1 of 2:
> -
> Chain start time: Sun Jun 11 00:32:04 2017
> Chain end time: Sun Jun 18 00:32:04 2017
> Number of contained backup sets: 4
> Total number of contained volumes: 8792
>  Type of backup set:Time:  Num volumes:
> Full Sun Jun 11 00:32:04 2017  6032
>  Incremental Thu Jun 15 00:32:03 2017  2543
>  Incremental Sat Jun 17 00:32:04 2017   153
>  Incremental Sun Jun 18 00:32:04 201764
> -
> 
> Secondary chain 2 of 2:
> -
> Chain start time: Mon Jun 19 00:32:03 2017
> Chain end time: Sun Jun 25 00:32:05 2017
> Number of contained backup sets: 3
> Total number of contained volumes: 8669
>  Type of backup set:Time:  Num volumes:
> Full Mon Jun 19 00:32:03 2017  5582
>  Incremental Fri Jun 23 00:32:04 2017  2808
>  Incremental Sun Jun 25 00:32:05 2017   279
> -
> 
> 
> Found primary backup chain with matching signature chain:
> -
> Chain start time: Mon Jun 26 00:32:05 2017
> Chain end time: Thu Jul 27 00:32:05 2017
> Number of contained backup sets: 26
> Total number of contained volumes: 11512
>  Type of backup set:Time:  Num volumes:
> Full Mon Jun 26 00:32:05 2017  1633
>  Incremental Wed Jun 28 00:32:30 2017  1821
>  Incremental Fri Jun 30 00:32:05 2017  4707
>  Incremental Tue Jul  4 00:32:04 2017   534
>  Incremental Wed Jul  5 00:32:04 2017   109
>  Incremental Thu Jul  6 00:32:04 2017   116
>  Incremental Fri Jul  7 00:32:04 2017   150
>  Incremental Sat Jul  8 00:32:06 2017   122
>  Incremental Sun Jul  9 00:32:04 2017   103
>  Incremental Mon Jul 10 00:32:04 2017   118
>  Incremental Tue Jul 11 00:32:04 2017   446
>  Incremental Wed Jul 12 00:32:05 201776
>  Incremental Thu Jul 13 00:32:04 201779
>  Incremental Fri Jul 14 00:32:05 201778
>  Incremental Sat Jul 15 00:32:03 201780
>  Incremental Sun Jul 16 00:32:05 201770
>  Incremental Mon Jul 17 00:32:04 201783
>  Incremental Tue Jul 18 00:32:04 2017   392
>  Incremental Thu Jul 20 00:32:03 2017   119
>  Incremental Fri Jul 21 00:32:03 201792
>  Incremental Sat Jul 22 00:32:04 201786
>  Incremental Sun Jul 23 00:32:04 201777
>  Incremental Mon Jul 24 00:32:04 201780
>  Incremental Tue Jul 25 00:32:04 201797
>  Incremental Wed Jul 26 00:32:06 2017   120
>  Incremental Thu Jul 27 00:32:05 2017   124
> -
> No orphaned or incomplete backup sets found.
> --- Finished state OK at 14:15:08.826 - Runtime 00:00:12.894 ---
> 
> 
> So I would expect that I can take the value under "num volumes" for the full 
> backups and divide them by 250 to get a number close to 3600, but I don't.

correct
 
> 1) why is this number so far off from what I expect?

compression? for the current primary chain i have no explanation though.
it should have roughly the same amount volumes as the previous two
provided that the data size was similar and big files didn't become
better compressible somehow.

&

Re: [Duplicity-team] [Question #655698]: Duplicity restore with selected files

[edso]

Question #655698 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/655698

Status: Open => Answered

edso proposed the following answer:
On 10.08.2017 06:44, Sivanath wrote:
> New question #655698 on Duplicity:
> https://answers.launchpad.net/duplicity/+question/655698
> 
> Is there any option for duplicity restore with selected files from large 
> backup set?
> 

you can use the --file-to-restore  option to restore one file or folder 
from the backup
  http://duplicity.nongnu.org/duplicity.1.html

afaics in the source --file-to-restore does not support multiple
entries.

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #654413]: Feature request: back up some files as full every time

[edso]

Question #654413 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/654413

Status: Open => Answered

edso proposed the following answer:
On 31.07.2017 22:03, Frank Bicknell wrote:
> New question #654413 on Duplicity:
> https://answers.launchpad.net/duplicity/+question/654413
> 
> Some files, like virtual machine images, are very large and take a really 
> long time (many hours) to back up incrementally when the file has changed 
> since the last backup.
> 
> Would it be possible to somehow flag certain files as "full backup only" 
> instead of having duplicity try to wedge in the changes to an incremental 
> backup?
> 
> The rest of the backup would still be incremental, but flagged files would 
> just get a full copy every time.
> 
> Seems to me this would apply in some circumstances.
> 

hey Frank,

the idea if incrementals itself is meant to speed up the backup
(duplicity's librsync eg. uses the rolling checksum
https://rsync.samba.org/tech_report/node3.html).

can you tell why you assume that backing up fully would be faster? did
you do tests?

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #654413]: Feature request: back up some files as full every time

[edso]

Question #654413 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/654413

edso proposed the following answer:
hey Frank,

On 01.08.2017 17:09, Frank Bicknell wrote:
> Question #654413 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/654413
> 
> Frank Bicknell posted a new comment:
> I occasionally find duplicity still running from the wee hours backup
> and it's working on backing up one of those virtualbox machine images
> that changed the day before. I don't use my virtual Windows box(es)
> every day, so it's a marked contrast to "normal" incremental backup
> days.

ic

> I assumed it was trying to reconcile changes in a huge, encrypted file.
> It always finishes, but there's always a lot of cpu utilization (not to
> the point of choking the machine, mind you).
> 
> Just seemed to me that if my premise is correct, it might make sense to
> back up such files full.

yeah, if. why do you assume that the full would be faster? :)
 
> It would obviously take up more space at the backend, but it would be
> faster.

but why? the librsync algorithm is designed to be as fast as a full 
passthrough, but only delivering the changes.
 
> I fully admit my findings are subjective and could be improved on.
> 
> If no one else agrees, then consider it just a passing thought and we
> can all get on with our lives.
> 

not so fast cowboy ;). how about the next time this happens you remember
the file and do a test?

first backup the whole file to a file:// location.
then do an incremental to that same location.
compare both times.
if you want you could change some stuff in the vm and run the incremental 
again, so that you effectively have changes to backup.

not saying you are not right. just saying, it shouldn't be so and of
course where's the data (proof;) ..ede/duply.net

PS: https://www.ted.com/talks/talithia_williams_own_your_body_s_data

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #631423]: Too many open files (again)

[edso]

Question #631423 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/631423

Status: Open => Answered

edso proposed the following answer:
Howie, i don't see the attachment?! ..ede/duply.net

On 10.05.2017 22:57, Howard Kaye wrote:
> Question #631423 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/631423
> 
> Howard Kaye gave more information on the question:
> The attached patch seems to fix the problem and keeps current behavior with
> respect to temp directories.
> 
> Howie
> 
> 
> On Tue, May 9, 2017 at 5:08 PM, edso <question631...@answers.launchpad.net>
> wrote:
> 
>> Your question #631423 on Duplicity changed:
>> https://answers.launchpad.net/duplicity/+question/631423
>>
>> edso proposed the following answer:
>> hey guys,
>>
>> that looks wrong. using os.tmpfile() may resolve the issue, but it
>> removes the users control, where the temp files are created. this comes
>> up on list list regularly when user's small temp partitions overflow and
>> they ask why duplicity is not respecting their temp dir setting.
>>
>> the python docs say
>>
>> https://docs.python.org/2/library/os.html#os.tmpfile
>> "
>> ... will be automatically deleted once there are no file descriptors for
>> the file.
>> "
>> https://docs.python.org/2/library/tempfile.html#tempfile.TemporaryFile
>> "
>> ... It will be destroyed as soon as it is closed (including an implicit
>> close when the object is garbage collected). ...
>> "
>>
>> Howard, could you please test if explicitly closing obsolete
>> TemporaryFiles instead in patchdir.py solves the issue as well?
>>
>> the whole condition looks fishy anyway. it's tempfile modules job to
>> take care of crossplatform compatibility, no need for us to treat some
>> platforms different than others. ..ede/duply.net
>>
>> On 09.05.2017 21:03, Howard Kaye wrote:
>>> Question #631423 on Duplicity changed:
>>> https://answers.launchpad.net/duplicity/+question/631423
>>>
>>> Status: Needs information => Open
>>>
>>> Howard Kaye gave more information on the question:
>>> The problem seems to be something going on in the tempfile Python
>> library.
>>> The following 1 line patch fixes the issue.
>>>
>>> *** /Users/howie/duplicity-0.7.10/duplicity/patchdir.py 2015-09-10
>>> 09:03:43.0 -0400
>>> --- patchdir.py 2017-05-09 14:27:38.0 -0400
>>> ***
>>> *** 496,502 
>>>   See https://bugs.launchpad.net/duplicity/+bug/670891 for
>>> discussion
>>>   of os.tmpfile() vs tempfile.TemporaryFile() w.r.t. Windows
>> /
>>> Linux.
>>>   """
>>> ! if sys.platform.startswith(('cygwin', 'windows')):
>>>   tempfp = os.tmpfile()
>>>   else:
>>>   tempfp =
>>> tempfile.TemporaryFile(dir=tempdir.default().dir())
>>> --- 496,502 
>>>   See https://bugs.launchpad.net/duplicity/+bug/670891 for
>>> discussion
>>>   of os.tmpfile() vs tempfile.TemporaryFile() w.r.t. Windows
>> /
>>> Linux.
>>>   """
>>> ! if sys.platform.startswith(('cygwin', 'windows',
>> 'darwin')):
>>>   tempfp = os.tmpfile()
>>>   else:
>>>   tempfp =
>>> tempfile.TemporaryFile(dir=tempdir.default().dir())
>>>
>>>
>>> On Tue, May 9, 2017 at 11:47 AM, Kenneth Loafman <
>>> question631...@answers.launchpad.net> wrote:
>>>
>>>> Your question #631423 on Duplicity changed:
>>>> https://answers.launchpad.net/duplicity/+question/631423
>>>>
>>>> Status: Open => Needs information
>>>>
>>>> Kenneth Loafman requested more information:
>>>> Part of the problem is the number of gpg processes, you should have a
>>>> max of 2 or 3, not 13.  When you installed the current version, was
>>>> there perhaps an older version on the system, maybe from ports or
>>>> homebrew?  If so, did you uninstall that version?  The repo versions
>>>> have different paths than the tarball version.
>>>>
>>>> To double check, run "locate GnuPGInterface.py".  If you have it, you
>>>> probably have a very old duplicity hanging out there and will need to
>>

Re: [Duplicity-team] [Question #631423]: Too many open files (again)

[edso]

Question #631423 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/631423

edso proposed the following answer:
hey guys,

that looks wrong. using os.tmpfile() may resolve the issue, but it
removes the users control, where the temp files are created. this comes
up on list list regularly when user's small temp partitions overflow and
they ask why duplicity is not respecting their temp dir setting.

the python docs say

https://docs.python.org/2/library/os.html#os.tmpfile
"
... will be automatically deleted once there are no file descriptors for the 
file.
"
https://docs.python.org/2/library/tempfile.html#tempfile.TemporaryFile
"
... It will be destroyed as soon as it is closed (including an implicit close 
when the object is garbage collected). ...
"

Howard, could you please test if explicitly closing obsolete
TemporaryFiles instead in patchdir.py solves the issue as well?

the whole condition looks fishy anyway. it's tempfile modules job to
take care of crossplatform compatibility, no need for us to treat some
platforms different than others. ..ede/duply.net

On 09.05.2017 21:03, Howard Kaye wrote:
> Question #631423 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/631423
> 
> Status: Needs information => Open
> 
> Howard Kaye gave more information on the question:
> The problem seems to be something going on in the tempfile Python library.
> The following 1 line patch fixes the issue.
> 
> *** /Users/howie/duplicity-0.7.10/duplicity/patchdir.py 2015-09-10
> 09:03:43.0 -0400
> --- patchdir.py 2017-05-09 14:27:38.0 -0400
> ***
> *** 496,502 
>   See https://bugs.launchpad.net/duplicity/+bug/670891 for
> discussion
>   of os.tmpfile() vs tempfile.TemporaryFile() w.r.t. Windows /
> Linux.
>   """
> ! if sys.platform.startswith(('cygwin', 'windows')):
>   tempfp = os.tmpfile()
>   else:
>   tempfp =
> tempfile.TemporaryFile(dir=tempdir.default().dir())
> --- 496,502 
>   See https://bugs.launchpad.net/duplicity/+bug/670891 for
> discussion
>   of os.tmpfile() vs tempfile.TemporaryFile() w.r.t. Windows /
> Linux.
>   """
> ! if sys.platform.startswith(('cygwin', 'windows', 'darwin')):
>   tempfp = os.tmpfile()
>   else:
>   tempfp =
> tempfile.TemporaryFile(dir=tempdir.default().dir())
> 
> 
> On Tue, May 9, 2017 at 11:47 AM, Kenneth Loafman <
> question631...@answers.launchpad.net> wrote:
> 
>> Your question #631423 on Duplicity changed:
>> https://answers.launchpad.net/duplicity/+question/631423
>>
>> Status: Open => Needs information
>>
>> Kenneth Loafman requested more information:
>> Part of the problem is the number of gpg processes, you should have a
>> max of 2 or 3, not 13.  When you installed the current version, was
>> there perhaps an older version on the system, maybe from ports or
>> homebrew?  If so, did you uninstall that version?  The repo versions
>> have different paths than the tarball version.
>>
>> To double check, run "locate GnuPGInterface.py".  If you have it, you
>> probably have a very old duplicity hanging out there and will need to
>> uninstall and purge it.
>>
>> --
>> To answer this request for more information, you can either reply to
>> this email or enter your reply at the following page:
>> https://answers.launchpad.net/duplicity/+question/631423
>>
>> You received this question notification because you asked the question.
>>
>

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #631423]: Too many open files (again)

[edso]

Question #631423 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/631423

edso posted a new comment:
nice catch Howie!

wonder why this worked with os.tmpfile() but not with
tempfile.TemporaryFile() before though.. ede/duply.net

On 16.05.2017 20:33, Howard Kaye wrote:
> Question #631423 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/631423
> 
> Status: Answered => Solved
> 
> Howard Kaye confirmed that the question is solved:
> OK, I found and fixed the root cause of this error:
> 
> Duplicity was failing in the glue code (in C), between Python and librsync.
> It did an fdopen() on the file descriptor from a Python File object, and then 
> never
> closed the File * which was returned.  Even though the Python File object 
> (and the
> file descriptor shared by the File * and the Python File) got closed when it 
> was
> deallocated, the File *'s were being leaked, and eventually the stdio library 
> ran
> out of File *'s (even though the number of open files was not that large).
> 
> The fix is to dup the file descriptor, and then close the file in the 
> deallocator
> routine in the glue code.  Duping the file lets the C code and the Python code
> each close the file, when they are done with it.
> 
> *** _librsyncmodule.c.orig  2017-01-06 09:23:21.0 -0500
> --- _librsyncmodule.c   2017-05-16 11:12:58.0 -0400
> ***
> *** 23,28 
> --- 23,29 
>* --- 
> */
>   
>   #include 
> + #include 
>   #include 
>   #define RS_JOB_BLOCKSIZE 65536
>   
> ***
> *** 287,292 
> --- 288,294 
> PyObject_HEAD
> rs_job_t *patch_job;
> PyObject *basis_file;
> +   FILE *cfile;
>   } _librsync_PatchMakerObject;
>   
>   /* Call with the basis file */
> ***
> *** 296,302 
> _librsync_PatchMakerObject* pm;
> PyObject *python_file;
> int fd;
> -   FILE *cfile;
>   
> if (!PyArg_ParseTuple(args, "O:new_patchmaker", _file))
>   return NULL;
> --- 298,303 
> ***
> *** 305,318 
>   PyErr_SetString(PyExc_TypeError, "Need true file object");
>   return NULL;
> }
> Py_INCREF(python_file);
>   
> pm = PyObject_New(_librsync_PatchMakerObject, &_librsync_PatchMakerType);
> if (pm == NULL) return NULL;
>   
> pm->basis_file = python_file;
> !   cfile = fdopen(fd, "rb");
> !   pm->patch_job = rs_patch_begin(rs_file_copy_cb, cfile);
>   
> return (PyObject*)pm;
>   }
> --- 306,327 
>   PyErr_SetString(PyExc_TypeError, "Need true file object");
>   return NULL;
> }
> +   /* get our own private copy of the file, so we can close it later. */
> +   fd = dup(fd);
> +   if (fd == -1) {
> + char buf[256];
> + strerror_r(errno, buf, sizeof(buf));
> + PyErr_SetString(PyExc_TypeError, buf);
> + return NULL;
> +   }
> Py_INCREF(python_file);
>   
> pm = PyObject_New(_librsync_PatchMakerObject, &_librsync_PatchMakerType);
> if (pm == NULL) return NULL;
>   
> pm->basis_file = python_file;
> !   pm->cfile = fdopen(fd, "rb");
> !   pm->patch_job = rs_patch_begin(rs_file_copy_cb, pm->cfile);
>   
> return (PyObject*)pm;
>   }
> ***
> *** 323,328 
> --- 332,340 
> _librsync_PatchMakerObject *pm = (_librsync_PatchMakerObject *)self;
> Py_DECREF(pm->basis_file);
> rs_job_free(pm->patch_job);
> +   if (pm->cfile) {
> + fclose(pm->cfile);
> +   }
> PyObject_Del(self);
>   }
>

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #631423]: Too many open files (again)

[edso]

Question #631423 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/631423

edso posted a new comment:
Howie, Ken,

would you agree then, that the whole os.tmpfile() exception

http://bazaar.launchpad.net/~duplicity-team/duplicity/0.8-series/view/head:/duplicity/patchdir.py#L498
"  
if sys.platform.startswith(('cygwin', 'windows', 'darwin')):
  tempfp = os.tmpfile()
else:
"

can be removed after the librsync adapter is patched?.. ede

On 17.05.2017 12:27, Howard Kaye wrote:
> Question #631423 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/631423
> 
> Howard Kaye posted a new comment:
> I think it just moved the error somewhere else.
> 
> On May 17, 2017 4:09 AM, "edso" <question631...@answers.launchpad.net>
> wrote:
> 
>> Your question #631423 on Duplicity changed:
>> https://answers.launchpad.net/duplicity/+question/631423
>>
>> edso posted a new comment:
>> nice catch Howie!
>>
>> wonder why this worked with os.tmpfile() but not with
>> tempfile.TemporaryFile() before though.. ede/duply.net
>>
>> On 16.05.2017 20:33, Howard Kaye wrote:
>>> Question #631423 on Duplicity changed:
>>> https://answers.launchpad.net/duplicity/+question/631423
>>>
>>> Status: Answered => Solved
>>>
>>> Howard Kaye confirmed that the question is solved:
>>> OK, I found and fixed the root cause of this error:
>>>
>>> Duplicity was failing in the glue code (in C), between Python and
>> librsync.
>>> It did an fdopen() on the file descriptor from a Python File object, and
>> then never
>>> closed the File * which was returned.  Even though the Python File
>> object (and the
>>> file descriptor shared by the File * and the Python File) got closed
>> when it was
>>> deallocated, the File *'s were being leaked, and eventually the stdio
>> library ran
>>> out of File *'s (even though the number of open files was not that
>> large).
>>>
>>> The fix is to dup the file descriptor, and then close the file in the
>> deallocator
>>> routine in the glue code.  Duping the file lets the C code and the
>> Python code
>>> each close the file, when they are done with it.
>>>
>>> *** _librsyncmodule.c.orig  2017-01-06 09:23:21.0 -0500
>>> --- _librsyncmodule.c   2017-05-16 11:12:58.0 -0400
>>> ***
>>> *** 23,28 
>>> --- 23,29 
>>>* ---
>> */
>>>
>>>   #include 
>>> + #include 
>>>   #include 
>>>   #define RS_JOB_BLOCKSIZE 65536
>>>
>>> ***
>>> *** 287,292 
>>> --- 288,294 
>>> PyObject_HEAD
>>> rs_job_t *patch_job;
>>> PyObject *basis_file;
>>> +   FILE *cfile;
>>>   } _librsync_PatchMakerObject;
>>>
>>>   /* Call with the basis file */
>>> ***
>>> *** 296,302 
>>> _librsync_PatchMakerObject* pm;
>>> PyObject *python_file;
>>> int fd;
>>> -   FILE *cfile;
>>>
>>> if (!PyArg_ParseTuple(args, "O:new_patchmaker", _file))
>>>   return NULL;
>>> --- 298,303 
>>> ***
>>> *** 305,318 
>>>   PyErr_SetString(PyExc_TypeError, "Need true file object");
>>>   return NULL;
>>> }
>>> Py_INCREF(python_file);
>>>
>>> pm = PyObject_New(_librsync_PatchMakerObject,
>> &_librsync_PatchMakerType);
>>> if (pm == NULL) return NULL;
>>>
>>> pm->basis_file = python_file;
>>> !   cfile = fdopen(fd, "rb");
>>> !   pm->patch_job = rs_patch_begin(rs_file_copy_cb, cfile);
>>>
>>> return (PyObject*)pm;
>>>   }
>>> --- 306,327 
>>>   PyErr_SetString(PyExc_TypeError, "Need true file object");
>>>   return NULL;
>>> }
>>> +   /* get our own private copy of the file, so we can close it later. */
>>> +   fd = dup(fd);
>>> +   if (fd == -1) {
>>> + char buf[256];
>>> + strerror_r(errno, buf, sizeof(buf));
>>> + PyErr_SetString(PyExc_TypeError, buf);
>>> + return NULL;
>>> +   }
>>> Py_INCREF(python_file);
>>>
>>> pm = PyObject_New(_librsync_PatchMakerObject,
>> &_librsync_PatchMakerType);
>>>

Re: [Duplicity-team] [Question #640874]: Using lots of memory while backup with Duplicity

[edso]

Question #640874 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/640874

Status: Open => Answered

edso proposed the following answer:

On June 6, 2017 10:46:51 AM GMT+02:00, Abhijit Sasmal 
<question640...@answers.launchpad.net> wrote:
>Question #640874 on Duplicity changed:
>/duplicity/+question/640874
>
>Status: Answered => Open
>
>Abhijit Sasmal is still having a problem:
>Hi,
>
>1) Thanks for your response. As I am using Centos 6.9, I can not
>install
>the latest one v0.7.12 which is for Centos7.  

try install the tarball from http://duplicity.nongnu.org ..

there is a step by step howto on installing duplicity into a custom location 
under TIPS here
  http://duply.net/wiki/index.php/Duply-documentation

>Can I Downgrade the
>version to 0.6.22?

that will probably not help.

>
>2) I am using encrypted and compressed backups.

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #631423]: Too many open files (again)

[edso]

Question #631423 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/631423

edso proposed the following answer:
On 05.05.2017 20:13, Howard Kaye wrote:
> New question #631423 on Duplicity:
> https://answers.launchpad.net/duplicity/+question/631423
> 
> In testing a full restore, I am seeing an exception get raised for "Too many 
> open files" from within Path.open:
> 
> I am running on macOS 10.12.4, duplicity 0.7.10
> 
> I have raised the ulimit for the number of files up to 12000.  
> lsof shows there are only 89 open files
> sample shows a number of threads blocked on subprocesses (gpg processes)
> sysctl -a shows:
>   kern.maxfiles: 24576
>   kern.maxfilesperproc: 10240
>   kern.num_files: 5356
> 
> There are currently only 17 incrementals following my latest full.  We 
> usually do monthly fulls.
> 
> Is anyone else seeing this recently?
> 

there's going to be a fix in 0.7.13, replacing the use of os.system('cp ...') 
with the native python routine, which should reduce the open files count 
dramatically
  http://duplicity.nongnu.org/CHANGELOG

it's explained in more detail here
  https://bugs.launchpad.net/duplicity/+bug/1684312

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #658091]: Why does duplicity ask for passwd when --encrypt-key + --sign-key is used???

[edso]

Question #658091 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/658091

Status: Open => Answered

edso proposed the following answer:
On 9/18/2017 20:53, ardabro wrote:
> Question #658091 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/658091
> 
> Status: Answered => Open
> 
> ardabro is still having a problem:
> Does it mean, that duplicity will ask me for this "unnecessary" password when 
> I resume interrupted process?

no, it will ask you for the proper passphrase or it won't proceed,
because the old encryption does not match the new encryption.

> Does it mean that during a process of backing-up some of my files are
encrypted symmetrically for a while?

no.

> If it has to ask me for a password when resuming - why cannot ask me
for password to my private key?

it does.

> What the advantage of using temporary password over encrypting
everything asymmetrically from the beginning to the end?

there is no temp passphrase. just because you can give junk when it is
not neeed does not mean the junk is used.

> 
> Sorry, but I don't grasp this idea. It looks strange.

better now? ..ede/duply.net

PS: Ken?

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #657939]: How to check backup (increment) integrity?

[edso]

Question #657939 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/657939

edso posted a new comment:
On 9/20/2017 22:34, ardabro wrote:
> Question #657939 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/657939
> 
> Status: Answered => Solved
> 
> ardabro confirmed that the question is solved:
> The necessity of giving fake-source path for verification is very
> confusing idea but  OK. Thanks
> 

it's legacy. previously the restored files meta data was _always_ compared to 
the local source, hence the need for the parameter.
as i wrote, somebody needs to take care of it. nobody did so far.

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #658091]: Why does duplicity ask for passwd when --encrypt-key + --sign-key is used???

[edso]

Question #658091 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/658091

Status: Open => Answered

edso proposed the following answer:
On 9/20/2017 22:23, ardabro wrote:
> Question #658091 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/658091
> 
> Status: Answered => Open
> 
> ardabro is still having a problem:
>> as i said. encryption passphrase during backup (p1-4) is always requested 
>> but only _needed_ when the backup 
>> resumes or the archive needs to be synced, because only in these cases 
>> decryption is needed during backup.
> 
>>> Does it mean, that duplicity will ask me for this "unnecessary"
> password when I resume interrupted process?
> 
>> no, it will ask you for the proper passphrase or it won't proceed, because 
>> the old encryption does not match 
>> the new encryption.
> 
> Sorry, but I't seems to be false.
> I started backup. Gave encryption password=="12345". I iterrupted the process 
> after 2nd volume.
> Then I restarted with exactly the same commandline. It realized that the 
> process was interrupted.
> It asked for encruption password and I gave "qwerty" this time. Everything 
> went perfectly OK.
> 
> A piece of log from 2nd run:
> Local and Remote metadata are synchronized, no sync needed.
> Last full backup left a partial set, restarting.
> Last full backup date: Wed Sep 20 22:01:48 2017
> GnuPG passphrase for decryption: 
> GnuPG passphrase for signing key: 
> RESTART: Volumes 2 to 2 failed to upload before termination.
>  Restarting backup at volume 2.
> Restarting after volume 1, file debian9.iso, block 1595
> ...
> 
> I still don't know when this password is necessary, why it asks only
> once (what if I make a typo?) and why it needs to verify anything with
> additional password when I order it to enrypt with my gpg key.
> 

this source code comment explains it.
  
http://bazaar.launchpad.net/~duplicity-team/duplicity/0.8-series/view/head:/bin/duplicity#L347

why the error is not raised is beyond me.

another thing you can try is doing an encrypted backup, then deleting the 
corresponding local archive dir (usually under ~/.cache/duplicity/).
on the next run it should try to synchronize the local w/ the remote and see 
the local is missing and download/decrypt some files from the remote to the 
local archive dir.
give the wrong passphrase this should fail.

what is your gpg version? make sure to kill the gpg-agent instance
between runs to make sure that really no passphrases are cached.

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #658091]: Why does duplicity ask for passwd when --encrypt-key + --sign-key is used???

[edso]

Question #658091 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/658091

Status: Open => Answered

edso proposed the following answer:
ardabro,

your command line looks like
> Args: /usr/bin/duplicity full --name=test_backup --encrypt-key=DEADBEEF 
> --sign-key=DEADBEEF /home/ard/temp/duplicity_test/src 
> file:///home/ard/temp/duplicity_test/dst

so you
1. sign
&
2. encrypt

when signing and encryption key are identical, the decryption passphrase
is (re)used for signing.

even when not signing, duplicity will ask the passphrase as a
precaution, it _might_ need to decrypt files for the repository (on
resume, archive folder sync).

> I'm able to decrypt with gpg key without entering any password (third
confusion)

that cannot be right, unless your private key is not protected by a
passphrase.

please read up on how gpg and asymmetrical encryption work in general.
the duplicity answer section is not the proper place to educate you in
this regard.

have fun ..ede/duply.net

On 9/16/2017 14:23, ardabro wrote:
> Question #658091 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/658091
> 
> Status: Answered => Open
> 
> ardabro is still having a problem:
> OK
> I'm Sorry, my mistake and additional confusion from gpg agent and cached keys.
> Everything is ok with gpg and its keys/passwords but I still have no
> idea why duplicity asks for decryption password when encryption key is used.
> It asks only once (second confusion) and does not use it at all -
> I'm able to decrypt with gpg key without entering any password (third 
> confusion)
> 
> 
> Main action: full
> 
> duplicity 0.7.11 (December 31, 2016)
> Args: /usr/bin/duplicity full --name=test_backup --encrypt-key=DEADBEEF 
> --sign-key=DEADBEEF /home/ard/temp/duplicity_test/src 
> file:///home/ard/temp/duplicity_test/dst
> Linux t430-deb 4.9.0-3-amd64 #1 SMP Debian 4.9.30-2+deb9u3 (2017-08-06) 
> x86_64 
> /usr/bin/python 2.7.13 (default, Jan 19 2017, 14:48:08) 
> [GCC 6.3.0 20170118]
> 
> 
> Last full backup date: none
> PASSPHRASE variable not set, asking user.
> GnuPG passphrase for decryption:# "12345" - doesn't 
> matter 
> PASSPHRASE variable not set, asking user.
> GnuPG passphrase for signing key:   # correct - my real 
> key-wrapping password is reqiured
> 
> --[ Backup Statistics ]--
> StartTime 1505562881.09 (Sat Sep 16 13:54:41 2017)
> 
> TotalDestinationSizeChange 912 (912 bytes)
> Errors 0
> -
> 
>> cd /home/ard/temp/duplicity_test/dst
>> ls -1
> duplicity-full.20170916T115427Z.manifest.gpg
> duplicity-full.20170916T115427Z.vol1.difftar.gpg
> duplicity-full-signatures.20170916T115427Z.sigtar.gpg
> 
>> gpg --output xxx --decrypt 
>> duplicity-full-signatures.20170916T115427Z.sigtar.gpg
> Please enter the passphrase to unlock the OpenPGP secret key:
> "ard <a...@gmail.com>"
> 2048-bit RSA key, ID ,
> created 2017-07-29 (main key ID ).
> 
> Passphrase: 
> gpg: encrypted with 2048-bit RSA key, ID , created 2017-07-29
>   "ard <a...@gmail.com>"
> gpg: Signature made Sat 16 Sep 2017 13:54:41 CEST
> gpg:using RSA key 
> gpg: Good signature from "ard <a...@gmail.com>" [ultimate]
>
> It asks for wrapping password for my keys; this is ok;
> and does NOT ask for this confusing duplicity password ("12345")
> I think it is a bug.
>

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #658091]: Why does duplicity ask for passwd when --encrypt-key + --sign-key is used???

[edso]

Question #658091 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/658091

Status: Open => Answered

edso proposed the following answer:
On 9/15/2017 22:14, ardabro wrote:
> Question #658091 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/658091
> 
> ardabro gave more information on the question:
> No I see that it is even worse:
> It asks for password ONCE, then asks for "passphrase for signing key" (ONCE).
> I can provide two different values, but they both doesn't matter. I'm able to 
> decrypt *.gpg files without entering any passwords.
> Who can explain it?
> 

that makes no sense..

1. what is your duplicity version?

2. what is your duplicity cmd line?

3. what is the complete terminal output? (obfuscate anything deemed
private), zip it if big

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #658091]: Why does duplicity ask for passwd when --encrypt-key + --sign-key is used???

[edso]

Question #658091 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/658091

edso proposed the following answer:
On 9/15/2017 22:04, ardabro wrote:
> New question #658091 on Duplicity:
> https://answers.launchpad.net/duplicity/+question/658091
> 
> Hi
> When i use --encrypt-key= parameter everything is OK. duplicity uses 
> my key and doesn't ask for encryption password. But when I use it with both:  
> --encrypt-key=  --sign-key=### it asks for encryption password 
> again!
> Why is it so? I just want to sign files with gpg. What is this password for?
> 
> 

because signing uses your private key along w/ your passphrase to
validate the message you've sent! everybody might encrypt against your
public key but only you can sign w/ your private key! the magic of
asymmetric encryption.

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #657939]: How to check backup (increment) integrity?

[edso]

Question #657939 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/657939

Status: Open => Answered

edso proposed the following answer:
you can!

using a recent duplicity it will ignore  (whule still
needed)  and simply check if the backup is restorable.. ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #658091]: Why does duplicity ask for passwd when --encrypt-key + --sign-key is used???

[edso]

Question #658091 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/658091

Status: Open => Answered

edso proposed the following answer:
On 9/17/2017 18:09, ardabro wrote:
> Question #658091 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/658091
> 
> Status: Answered => Open
> 
> ardabro is still having a problem:
> Sorry, maybe my English is too bad. Also I see I introduced a bit mess in 
> this line:
> "I'm able to decrypt with gpg key without entering any password (third 
> confusion)"
> This is false. It asks for password that secures my GPG key(s), and ONLY for 
> this one.
> Actually it asks everytime because I have gpg keys caching disabled.
> So, when decrypting, the behaviour is 100% CORRECT for scenario with gpg keys.
> The problem occurs when backup is created.
> 
> Once again the whole case (PLEASE, READ CAREFULLY):
> 
> 1) run duplicity with optios as previously
> 2) provide ___ANY_RANDOM_UNNECESSARY_USELESS_PASSWORD__ after "GnuPG 
> passphrase for decryption:"   (only once, no confirmation required!)
> 3) provide __REAL__ password that wraps my BOTH gpg keys (it is actually the 
> same key pair) after "GnuPG passphrase for signing key:"
> 4) backup is created.
> 5) run gpg --output xxx --decrypt duplicity-full*.gpg ORduplicity 
> restore file://dst trg
> 6) provide __REAL__ password that wraps my both keys (the same one as used in 
> p3)
> 7) file is decrypted or backup restored WITHOUT using this 
> ___ANY_RANDOM_UNNECESSARY_USELESS_PASSWORD__ used in p2 !
> 
> The above scenario is 100% reproductible!
> 
> P.S.
> In p2) duplicity asks the for the password with exactly the same way as it 
> does it twice when I encrypt with password directly (no keys).
> But this time it doesn't use entered value at all (why should it use it when 
> a gpg-key is used for encryption?)
> 

as i said. encryption passphrase during backup (p1-4) is always
requested but only _needed_ when the backup resumes or the archive needs
to be synced, because only in these cases decryption is needed during
backup.

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #656947]: restoring metadata from amazon glacier

[edso]

Question #656947 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/656947

edso proposed the following answer:
On 8/29/2017 21:33, Martin wrote:
> Question #656947 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/656947
> 
> Martin posted a new comment:
> Of course you can replace every occurence of key2 with 
> "self.bucket.get_key(key.key)".
> And every occurence of key can be replaced with self._listed_keys[key_name].
> So key2 can be replaced by 
> self.bucket.get_key(self._listed_keys[key_name].key).
> Or self.bucket.get_key(self._listed_keys[self.key_prefix + 
> remote_filename].key).
> But this is not faster and does not save any memory.
> 

well, where is your objection then? no reason to create a variable that
is only used once is all i am saying. ..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #657939]: How to check backup (increment) integrity?

[edso]

Question #657939 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/657939

Status: Open => Answered

edso proposed the following answer:
On 9/12/2017 22:58, ardabro wrote:
> New question #657939 on Duplicity:
> https://answers.launchpad.net/duplicity/+question/657939
> 
> Is there any way to check integrity of backup files?
> I don't mean verification with comparison to original files.

verify does not compare to the source files since some time unless you
give he --compare-data parameter.

> I mean integrity test without touching source folder, i.e check whether any 
> backup file is corrupted.
> The only way that came to my mind is restore with --dry-run but it'll check 
> the whole backup and may last for ages.
> How to check integrity of certain increment? Is there any possibility?

no. as increments depend on earlier backups all those will be used as
well. from the man page

"
verify [--compare-data] [--time ] [--file-to-restore ]  

Restore backup contents temporarily file by file and compare against the local 
path’s contents. duplicity will exit with a non-zero error level if any files 
are different. On verbosity level info (4) or higher, a message for each file 
that has changed will be logged. 
The --file-to-restore option restricts verify to that file or folder. The 
--time option allows to select a backup to verify against. The --compare-data 
option enables data comparison (see below).
"

it should be updated that data comparison is not the default anymore although 
the local_path is still obligatory. somebody needs to find the time to clean 
that out
.

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #658653]: Cannot allocate memory with large manifest file

[edso]

Question #658653 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/658653

edso proposed the following answer:
Siva,

are you sure that this is caused by the version upgrade? maybe it is
coincidental? i would try two things.

1. make sure that nothing lingers in your ram and there is plenty memory
free before you run duplicity.

2. if that doesn't help, downgrade to the version of duplicity you used
before and see if the error vanishes.

what's your distro? what version of duplicity did you use before?

..ede/duply.net


On 26.09.2017 09:15, Sivanath wrote:
> Question #658653 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/658653
> 
> Description changed to:
> Hi team
> 
> I have used the duplicity for backup my server files from few months
> before.  And It was working fine. Currently I have upgraded the
> duplicity new version(0.7.14),  after upgraded the backup manifest cache
> file written in large size(near to GBs) due to bulk delete or changes in
> files.
> 
> It was stopped the next backup process with this error - "OSError:
> [Errno 12] Cannot allocate memory".
> 
> Please help me on this. Is there any way to solve this problem?
> 
> Herewith  I have placed the logged error information for your reference.
> 
> Traceback (most recent call last):
>   File "/usr/bin/duplicity", line 1548, in 
> with_tempdir(main)
>   File "/usr/bin/duplicity", line 1534, in with_tempdir
> fn()
>   File "/usr/bin/duplicity", line 1383, in main
> do_backup(action)
>   File "/usr/bin/duplicity", line 1515, in do_backup
> check_last_manifest(col_stats)  # not needed for full backup
>   File "/usr/bin/duplicity", line 1217, in check_last_manifest
> last_backup_set.check_manifests()
>   File "/usr/local/lib/python2.7/dist-packages/duplicity/collections.py", 
> line 208, in check_manifests
> remote_manifest = self.get_remote_manifest()
>   File "/usr/local/lib/python2.7/dist-packages/duplicity/collections.py", 
> line 241, in get_remote_manifest
> manifest_buffer = self.backend.get_data(self.remote_manifest_name)
>   File "/usr/local/lib/python2.7/dist-packages/duplicity/backend.py", line 
> 677, in get_data
> fin = self.get_fileobj_read(filename, parseresults)
>   File "/usr/local/lib/python2.7/dist-packages/duplicity/backend.py", line 
> 671, in get_fileobj_read
> return tdp.filtered_open_with_delete("rb")
>   File "/usr/local/lib/python2.7/dist-packages/duplicity/dup_temp.py", line 
> 120, in filtered_open_with_delete
> fh = FileobjHooked(path.DupPath.filtered_open(self, mode))
>   File "/usr/local/lib/python2.7/dist-packages/duplicity/path.py", line 779, 
> in filtered_open
> return gpg.GPGFile(False, self, gpg_profile)
>   File "/usr/local/lib/python2.7/dist-packages/duplicity/gpg.py", line 217, 
> in __init__
> 'logger': self.logger_fp})
>   File "/usr/local/lib/python2.7/dist-packages/duplicity/gpginterface.py", 
> line 374, in run
> create_fhs, attach_fhs)
>   File "/usr/local/lib/python2.7/dist-packages/duplicity/gpginterface.py", 
> line 414, in _attach_fork_exec
> process.pid = os.fork()
> OSError: [Errno 12] Cannot allocate memory
>

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #658832]: Problem with Dropbox-Backend

[edso]

Question #658832 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/658832

Status: Open => Answered

edso proposed the following answer:
On 02.10.2017 08:54, Luca Bertoncello wrote:
> New question #658832 on Duplicity:
> https://answers.launchpad.net/duplicity/+question/658832
> 
> Hi all
> 
> I'm using duplicity with the Dropbox-backend on some servers.
> All worked fine until two days ago, then I just got:
> 
> dpbx error: [400] u'v1_retired'
> Attempt 1 failed. ErrorResponse: [400] u'v1_retired'
> 
> So, I tried to update duplicity and the Dropbox-Backend, but I always get
> this error:
> 
> BackendException: dpbx: DPBX_APP_KEY environment variable not set
> 
> so I tried this http://duplicity.nongnu.org/duplicity.1.html#sect12 but it
> does not help...
> I exported DPBX_APP_KEY and DPBX_APP_SECRET, then I run duplicity and I got
> an URL to confirm the action.
> I confirmed it and I got an authorization code that I pasted in duplicity.
> Unfortunately I get:
> 
> BackendException: dpbx: Unable to obtain access token: Timeout value connect
> was Timeout(connect=None, read=None, total=None), but it must be an int or
> float.
> 
> Any idea how can I solve my problem?
> 

Luca,

try asking on the mailing list where some people supposedly use dropbox.
  https://lists.nongnu.org/mailman/listinfo/duplicity-talk

I'll cc Dmitry and Jno, who are listed as the original authors of the
backend, just in case.

good luck.. ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #658832]: Problem with Dropbox-Backend

[edso]

Question #658832 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/658832

Status: Open => Answered

edso proposed the following answer:
On 02.10.2017 11:53, Luca Bertoncello wrote:
> Question #658832 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/658832
> 
> Status: Answered => Open
> 
> Luca Bertoncello is still having a problem:
> Unfortunately I can't contact the ML, since the Mailserver always
> rejects my E-Mails...
> 

i remember, can't you use another email address? some freemailer gmail,
gmx, yahoo whatever?

sorry for the inconvenience. as i wrote before, complain to savannah and
see what they say ..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #658832]: Problem with Dropbox-Backend

[edso]

Question #658832 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/658832

Status: Open => Answered

edso proposed the following answer:
On 02.10.2017 12:44, Luca Bertoncello wrote:
> Question #658832 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/658832
> 
> Status: Answered => Open
> 
> Luca Bertoncello is still having a problem:
> Really, I don't have any other E-Mail, since my current address runs 
> perfectly...
> I'm surely don't create an account just for the buggy gnu.org...
> 

your decision. just saying your chances w/ this issue are better on the
ml and if i were you i would simply register one in a few minutes.

have fun. ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #658832]: Problem with Dropbox-Backend

[edso]

Question #658832 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/658832

edso proposed the following answer:
On 02.10.2017 08:54, Luca Bertoncello wrote:
> New question #658832 on Duplicity:
> https://answers.launchpad.net/duplicity/+question/658832
> 
> Hi all
> 
> I'm using duplicity with the Dropbox-backend on some servers.
> All worked fine until two days ago, then I just got:
> 
> dpbx error: [400] u'v1_retired'
> Attempt 1 failed. ErrorResponse: [400] u'v1_retired'
> 
> So, I tried to update duplicity and the Dropbox-Backend, but I always get
> this error:
> 
> BackendException: dpbx: DPBX_APP_KEY environment variable not set
> 
> so I tried this http://duplicity.nongnu.org/duplicity.1.html#sect12 but it
> does not help...
> I exported DPBX_APP_KEY and DPBX_APP_SECRET, then I run duplicity and I got
> an URL to confirm the action.
> I confirmed it and I got an authorization code that I pasted in duplicity.
> Unfortunately I get:
> 
> BackendException: dpbx: Unable to obtain access token: Timeout value connect
> was Timeout(connect=None, read=None, total=None), but it must be an int or
> float.
> 
> Any idea how can I solve my problem?
> 
> Thanks
> Luca Bertoncello
> 

there is the possibility that the backend might not work until it is updated to 
dropbox v2 API. just read this german news snippet
  
https://www.heise.de/newsticker/meldung/Dropbox-API-v1-abgeschaltet-3848199.html

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #452086]: Use with Oracle cloud

[edso]

Question #452086 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/452086

edso proposed the following answer:
On 9/27/2017 20:53, CJ Niemira wrote:
> Question #452086 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/452086
> 
> CJ Niemira proposed the following answer:
> I'm not terribly familiar with launchpad or the duplicity workflow, but
> here's a backend driver for Oracle BMC:
> 
> http://bazaar.launchpad.net/~siegeofnorth/duplicity/duplicity/view/head:/duplicity/backends/oraclebmcbackend.py
> 

hey CJ,

would you mind modifying the man page accordingly, especially sections 
REQUIREMENTS and URL FORMATS ?
  
http://bazaar.launchpad.net/~siegeofnorth/duplicity/duplicity/view/head:/bin/duplicity.1

thanks ..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #658653]: Cannot allocate memory with large manifest file

[edso]

Question #658653 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/658653

edso proposed the following answer:
Ken,

1.
can you pinpoint a version, when that happend? so we can suggest users to stick 
w/ it until this is solved in a future release?

2.
manifests are still one file right? not splitted like volumes on the backend.

..ede/duply.net

On 28.09.2017 12:19, Kenneth Loafman wrote:
> Question #658653 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/658653
> 
> Kenneth Loafman proposed the following answer:
> Yes, there was a change a long while back to add the list of changed files
> to the manifest.  This allowed us to do collection status with individual
> file tracking to see where the file had changed.  I think this should have
> been an optional list or in a separate database.  It seems to be a burden
> on small machines.
> 
> ...Ken
> 
> 
> On Thu, Sep 28, 2017 at 4:33 AM, edso <question658...@answers.launchpad.net>
> wrote:
> 
>> Question #658653 on Duplicity changed:
>> https://answers.launchpad.net/duplicity/+question/658653
>>
>> edso requested more information:
>> Siva,
>>
>> 1.
>> can you pastebin or put it somewhere online, examples (created w/
>> 0.7.08,13) on how manifests are suddenly bigger?
>>
>> 2.
>> can you show me that your ram is really empty between the full/incr  runs?
>>   https://www.cyberciti.biz/faq/linux-check-memory-usage/
>>
>> 3.
>> are there any lingering gpg processes still in memory between duplicity
>> runs?
>>   'ps -C gpg,gpg2'
>>
>> @Ken: any idea?
>>
>> ..ede/duply.net
>>
>> --
>> You received this question notification because your team duplicity-team
>> is an answer contact for Duplicity.
>>
>> ___
>> Mailing list: https://launchpad.net/~duplicity-team
>> Post to : duplicity-team@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~duplicity-team
>> More help   : https://help.launchpad.net/ListHelp
>>
>

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #658653]: Cannot allocate memory with large manifest file

[edso]

Question #658653 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/658653

Status: Open => Answered

edso proposed the following answer:
Siva,

also these tickets might be of help

https://answers.launchpad.net/duplicity/+question/270906
https://bugs.launchpad.net/bugs/1684312
https://bugs.launchpad.net/bugs/896728

long story short. start a new backup chain. limit the amount of
files/data so that, as memory usage grows with backup size/chain length.

..ede/duply.net

On 26.09.2017 09:15, Sivanath wrote:
> Question #658653 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/658653
> 
> Description changed to:
> Hi team
> 
> I have used the duplicity for backup my server files from few months
> before.  And It was working fine. Currently I have upgraded the
> duplicity new version(0.7.14),  after upgraded the backup manifest cache
> file written in large size(near to GBs) due to bulk delete or changes in
> files.
> 
> It was stopped the next backup process with this error - "OSError:
> [Errno 12] Cannot allocate memory".
> 
> Please help me on this. Is there any way to solve this problem?
> 
> Herewith  I have placed the logged error information for your reference.
> 
> Traceback (most recent call last):
>   File "/usr/bin/duplicity", line 1548, in 
> with_tempdir(main)
>   File "/usr/bin/duplicity", line 1534, in with_tempdir
> fn()
>   File "/usr/bin/duplicity", line 1383, in main
> do_backup(action)
>   File "/usr/bin/duplicity", line 1515, in do_backup
> check_last_manifest(col_stats)  # not needed for full backup
>   File "/usr/bin/duplicity", line 1217, in check_last_manifest
> last_backup_set.check_manifests()
>   File "/usr/local/lib/python2.7/dist-packages/duplicity/collections.py", 
> line 208, in check_manifests
> remote_manifest = self.get_remote_manifest()
>   File "/usr/local/lib/python2.7/dist-packages/duplicity/collections.py", 
> line 241, in get_remote_manifest
> manifest_buffer = self.backend.get_data(self.remote_manifest_name)
>   File "/usr/local/lib/python2.7/dist-packages/duplicity/backend.py", line 
> 677, in get_data
> fin = self.get_fileobj_read(filename, parseresults)
>   File "/usr/local/lib/python2.7/dist-packages/duplicity/backend.py", line 
> 671, in get_fileobj_read
> return tdp.filtered_open_with_delete("rb")
>   File "/usr/local/lib/python2.7/dist-packages/duplicity/dup_temp.py", line 
> 120, in filtered_open_with_delete
> fh = FileobjHooked(path.DupPath.filtered_open(self, mode))
>   File "/usr/local/lib/python2.7/dist-packages/duplicity/path.py", line 779, 
> in filtered_open
> return gpg.GPGFile(False, self, gpg_profile)
>   File "/usr/local/lib/python2.7/dist-packages/duplicity/gpg.py", line 217, 
> in __init__
> 'logger': self.logger_fp})
>   File "/usr/local/lib/python2.7/dist-packages/duplicity/gpginterface.py", 
> line 374, in run
> create_fhs, attach_fhs)
>   File "/usr/local/lib/python2.7/dist-packages/duplicity/gpginterface.py", 
> line 414, in _attach_fork_exec
> process.pid = os.fork()
> OSError: [Errno 12] Cannot allocate memory
>

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #656947]: restoring metadata from amazon glacier

[edso]

Question #656947 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/656947

Status: Open => Answered

edso proposed the following answer:
Martin,

there are multiple calls to backend.pre_process_download() in
bin/duplicity. so it might make sense to rework BackendWrapper
accordingly.

BackendWrapper is a kind of generalizing class for backends providing eg. 
put(), which is implemented in the backends as private _put() method. so 
1. adding a pre_process_download() to the wrapper, 
2. making it work with single entries and lists and 
3. renaming / running the underlying _pre_process_download() 
should be sufficient here. boto seems to be the only instance using it 
currently.

renaming the method to pre_get() or pre_process_get() would probably
streamline the general naming.

can you do the changes and generate a diff patch? or even better setup a
bazaar branch, it's not that difficult.

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #656947]: restoring metadata from amazon glacier

[edso]

Question #656947 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/656947

edso proposed the following answer:
On 25.08.2017 15:47, Martin wrote:
> Question #656947 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/656947
> 
> Martin posted a new comment:
> Also I have found the following problem:
> _boto_single assumes that objects in class "GLACIER" cannot be downloaded 
> ("if key.storage_class == "GLACIER":").
> This is not true.
> http://docs.aws.amazon.com/AmazonS3/latest/dev/restoring-objects.html states:
> After you receive a temporary copy of the restored object, the 
> object's storage class remains GLACIER 
> (a GET or HEAD request will return GLACIER as the storage class). 
> So it is not a good idea to force objects back to S3.
> 
> The correct way for _boto_single.pre_process_download to do it:
> 
> if key.storage_class == "GLACIER":
>   if key.ongoing_restore: wait or ignore
>   else if key.expiry_date: restore finished, temp copy available
>   else key.restore(days=2)
> 
>  See also http://boto.cloudhackers.com/en/latest/s3_tut.html
> 

good catch. again, you are welcome to fix those. i, unfortunately do not
use s3 nor have i the need to, but would be willing to help you get your
changes committed ;)

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #656947]: restoring metadata from amazon glacier

[edso]

Question #656947 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/656947

edso proposed the following answer:
On 25.08.2017 15:18, Martin wrote:
> Question #656947 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/656947
> 
> Martin posted a new comment:
> It is the first time I look here. I can start an editor and understand
> python, but I have no idea how to create patches and I don't know what a
> bazaar is :-)
> 

how about you use the latest sources as a start, patch/test them and
send me the result as a compressed archive? i could do the rest and
suggest it for merging if the changes look good.

how to run a devl duplicity is described in README-REPO.
  
http://bazaar.launchpad.net/~duplicity-team/duplicity/0.8-series/view/head:/README-REPO
it also describes howto download from bazaar. howto upload a bazaar branch is 
described on launchpad.

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #656947]: restoring metadata from amazon glacier

[edso]

Question #656947 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/656947

edso proposed the following answer:
On 25.08.2017 16:43, Martin wrote:
> Question #656947 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/656947
> 
> Martin posted a new comment:
> This is my new _boto_single.py.pre_process_download:
> 
> def pre_process_download(self, remote_filename, wait=False):
> # Used primarily to restore files in Glacier
> key_name = self.key_prefix + remote_filename
> if not self._listed_keys.get(key_name, False):
> self._listed_keys[key_name] = list(self.bucket.list(key_name))[0]
> key = self._listed_keys[key_name]
> key2 = self.bucket.get_key(key.key)  #why do we need key2?
> 
> if key2.storage_class == "GLACIER":
> if not key2.expiry_date: # no temp copy avail
> if not key2.ongoing_restore:
> log.Info("File %s is in Glacier storage, restoring" % 
> remote_filename)
> key.restore(days=2)  # Shouldn't need this again after 2 
> days
> if wait:
> log.Info("Waiting for file %s to restore in Glacier" % 
> remote_filename)
> while not key2.expiry_date:
>time.sleep(60)
>self.resetConnection()
> log.Info("File %s was successfully restored in Glacier" % 
> remote_filename)
> 

Martin,

the key2 business seems unnecessary. why don't you leave the method as
it is and simply add key.expiry_date as a condition for not restoring?
eg.

  # in deep freeze and _no_ temp copy available
  if key.storage_class == "GLACIER" and not 
self.bucket.get_key(key.key).expiry_date:

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #656947]: restoring metadata from amazon glacier

[edso]

Question #656947 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/656947

edso proposed the following answer:
On 25.08.2017 18:33, Martin wrote:
> Martin posted a new comment:
> It does not work with "key", it is necessary to create key2.

Martin, did you try my suggestion?

self.bucket.get_key(key.key).expiry_date

  is essentially the _same_ as

key2 = self.bucket.get_key(key.key) #why do we need key2?
key2.expiry_date

, just without adding a temporary variable!

fetching the new key is probably necessary because the connection is
rest in the loop.

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #659626]: S3 data transfer out - getObject requests

[edso]

Question #659626 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/659626

Status: Open => Answered

edso proposed the following answer:
On 19.10.2017 08:28, Jasper wrote:
> Question #659626 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/659626
> 
> Description changed to:
> I have Duplicity running nightly backups of 7 servers and I'm seeing
> very high S3 costs of $400 a month that I didn't expect.
> 
> After investigating with Amazon's cost explorer I noticed those costs
> are almost all due to the large amount of 'data transfer out, getObject
> requests'.
> 
> I'm not restoring any data, so why is Duplicity doing so many getObject
> requests to S3?
> 
> I noticed that I might have misconfigured the amount of backup cycles a
> bit, could that be the reason?
> 
> First my config was as follows:
> 
> MAX_AGE=2M
> MAX_FULL_BACKUPS=2
> MAX_FULLS_WITH_INCRS=1
> MAX_FULLBKP_AGE=1M
> VOLSIZE=250
> 
> I have now changed it to this:
> 
> MAX_AGE=6M
> MAX_FULL_BACKUPS=2
> MAX_FULLS_WITH_INCRS=1
> # Deactivated this
> #MAX_FULLBKP_AGE=1M
> VOLSIZE=250
> 

Jasper,

what is your duply command to run this conf? ..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Merge] lp:~crosser/duplicity/dpbx-fix-file-listing into lp:duplicity

[edso]

Review: Needs Fixing

hi,

that workaround looks fishy to me. it will also mask other ListFolderErrors, 
even ones with a proper cause, that the user needs to see to take appropriate 
action.

can you dig into the error and find out the detailed error, so you might only 
ignore this specific error?

another less invasive workaround would be placing a dummy file in the new 
folder during creation.

probably the correct solution would be raising a bug ticket over at
  https://github.com/dropbox/dropbox-sdk-python/issues

thanks!.. ede/duply.net
-- 
https://code.launchpad.net/~crosser/duplicity/dpbx-fix-file-listing/+merge/334274
Your team duplicity-team is subscribed to branch lp:duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

[Duplicity-team] [Merge] lp:~crosser/duplicity/dpbx-fix-file-listing into lp:duplicity

[edso]

The proposal to merge lp:~crosser/duplicity/dpbx-fix-file-listing into 
lp:duplicity has been updated.

Status: Needs review => Rejected

For more details, see:
https://code.launchpad.net/~crosser/duplicity/dpbx-fix-file-listing/+merge/334274
-- 
Your team duplicity-team is subscribed to branch lp:duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #660463]: Interest in refactoring dup_time.py

[edso]

Question #660463 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/660463

Status: Open => Answered

edso proposed the following answer:
On 11/11/2017 14:53, Konstantin Schubert wrote:
> Question #660463 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/660463
> 
> Konstantin Schubert gave more information on the question:
> 
> I've got a few more questions. If the mailing list isn't the right place for 
> them, let me know. 
> 
> First off, I want to say that I have high respect of code that is time
> proven and works, even if it's not written in the latest greatest
> language :)
> 
> That being said, I still think that I'd be able to make some careful
> contributions.
> 
> My first question: 
> Would it be acceptable to raise the minimum required python version to 2? 
> 
> Second question: 
> My impression is that the dup_time module is mainly used for parsing file 
> names into an internal time representation and vice versa. Therefore, no 
> matter what changes I make here, I'll have to be very careful not to break 
> backwards compatibility in parsing the file names. 
> 
> I'd probably start working by writing tests for all the file name formats 
> that need to be covered and then ask here on the list for confirmation that 
> the tests are exhaustive. 
> Afterwards, I can start work on refactoring the module by making use of 
> datetime and pytz. 
> 
> Does that sound reasonable?
> 

Konstantin,

it does. i am wondering tough why you want to rewrite a perfectly
functioning part of duplicity when there are bugs and feature request en
masse on launchpad already. just a tought.

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #660695]: duplicity fails with file too large error

[edso]

Question #660695 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/660695

Status: Open => Answered

edso proposed the following answer:
On 13.11.2017 16:57, Camille wrote:
>   File "/usr/lib/python2.7/gzip.py", line 236, in write
> self.fileobj.write( self.compress.compress(data) )
>   File "/usr/lib/python2.7/dist-packages/duplicity/gpg.py", line 419, in write
> result = self.fileobj.write(buf)
> IOError: [Errno 27] File too large
> 
> This backup is small (77MB) anf there are plenty of available space on both 
> sides, even on /tmp/. Manual ssh to the remote user/host works.
> 
> Any idea?

what file systems are involved locally. can you post the output of
'mount' ?

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #660704]: https://www.universe.com/events/bigg-boss-11-14th-november-2017-14th-nov-2017-watch-online-urdu-drama-tv-tickets-RVW37Y/

[edso]

Question #660704 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/660704

Status: Open => Invalid

edso rejected the question:
Spam

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #668566]: Can I encrypt an existing unencrypted backup?

[edso]

Question #668566 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/668566

Status: Open => Answered

edso proposed the following answer:
On 5/4/2018 0:17, nicholas wrote:
> New question #668566 on Duplicity:
> https://answers.launchpad.net/duplicity/+question/668566
> 
> Just curious if I can enable encryption after I've done an unencrypted backup.
> At the time I didn't want to enter a passphrase.
> 

no, sorry. all you can do is change parameters and start a new chain.
..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #668671]: Duplicity: Key Error 418

[edso]

Question #668671 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/668671

Status: Open => Answered

edso proposed the following answer:
On 5/6/2018 13:03, Rebecca wrote:
> Question #668671 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/668671
> 
> Status: Needs information => Open
> 
> Rebecca gave more information on the question:
> The command i used:
> duplicity --force -t 7D "file:///media/rebecca/My Passport/Temp_Backup" .
> 
> and the pastebin is here:
> 
> https://paste.ubuntu.com/p/KNfC3hxnKm/
> 

this looks incomplete. can you dump and post the _full_ output of your
command? raising log verbosity to debug might give more info as well.
eg.

duplicity --force -v9 -t 7D "file:///media/rebecca/My
Passport/Temp_Backup" . > /tmp/duperror.log 2>&1

this can get very big. so zipping the result and upload the archive to a
cloud/filehoster might be needed.

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #668671]: Duplicity: Key Error 418

[edso]

Question #668671 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/668671

Status: Open => Answered

edso proposed the following answer:
On 08.05.2018 13:33, Rebecca wrote:
> Question #668671 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/668671
> 
> Status: Answered => Open
> 
> Rebecca is still having a problem:
> Hi edso,
> 
> I have rerun using your command and have loaded the log to the google
> drive:
> 
> https://drive.google.com/open?id=1j78I7XS4VyGwayAKkGRRkhh0kp8i8JbO
> 
> I hope that helps.
> 

it does. two issues


1.
> Making directory .
> Writing home of type dir
> Making directory ./home
> Writing home/rebecca of type dir
> Error '[Errno 13] Permission denied: './home/rebecca'' processing home/rebecca
> Skipping home/rebecca/..profile.swp because of previous error
> Skipping home/rebecca/.AndroidStudio2.3 because of previous error

there is some issue w/ creating file in the current folder. choose a
different target folder.

2.
that's a fatal one unfortunately.

you are missing the file 'duplicity-
full.20180227T142615Z.vol418.difftar.gpg' (Key 418) on your backend, so
duplicity errs out when trying to process it.

you may try '--ignore-errors' as documented in the man page
  http://duplicity.nongnu.org/duplicity.1.html
, but there will be lost files unless you miraculously find and place the 
missing file on the backend again.

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #669105]: Error 219 at allwas at end backup

[edso]

Question #669105 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/669105

Status: Open => Answered

edso proposed the following answer:
On 16.05.2018 17:52, Battant wrote:
SNIP
> TotalDestinationSizeChange 0 (0 bytes)
> Errors 219
> 
> Question : what say end why the Error 219 ?
> 
> Is it a bug ?

this is from the summary. look through the _whole_ console output/log
and you will see that duplcity had problems to access or read files 219
times. solve these and the error count will go to zero.

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #670362]: sftp backend ssh options

[edso]

Question #670362 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/670362

edso proposed the following answer:
On 21.06.2018 10:57, edso wrote:
> paramiko is a python native ssh implementation and as such does not support 
> the same parameters

ok, that's misleading. the configuration itself is different while the options 
should be there for the most part. if you are interested you can see how the 
command line parameter is mapped to the paramiko option in
  
https://bazaar.launchpad.net/~duplicity-team/duplicity/0.8-series/view/head:/duplicity/backends/ssh_paramiko_backend.py#L196
.

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #668404]: Does duplicity modify atime?

[edso]

Question #668404 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/668404

Status: Open => Answered

edso proposed the following answer:
On 30.04.2018 02:52, Tim wrote:
> New question #668404 on Duplicity:
> https://answers.launchpad.net/duplicity/+question/668404
> 
> My data resides on a SSD and — thanks to Write Amplification — any 
> modification of an atime will result in not only the inode being modified, 
> but the whole block that it resides on being erased and rewritten. That is, 
> obviously, undesirable.
> 
> When Duplicity backs up files, does it modify the atime attribute of the 
> source files in the process?
> 

afaik. atime is a file system feature, which needs to be disabled in the
file system as a mount option if it is not wanted (noatime, nodiratime).
not sure what the implications for a system disk are though if you do
so.

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #668404]: Does duplicity modify atime?

[edso]

Question #668404 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/668404

Status: Open => Answered

edso proposed the following answer:
On 30.04.2018 11:22, Tim wrote:
> Question #668404 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/668404
> 
> Status: Answered => Open
> 
> Tim is still having a problem:
> Yes, atime is a filesystem feature.  But it is possible to
> programatically read and store the original atime, then read the file,
> then restore the original atime — using utimes() — such that atime
> effectively does not change.  I am wondering if Duplicity 'preserves'
> the original atime in such a manner.
> 

given the age of the original source i'd guess not. if you really want
to know you can easily check the sources yourself, or easier create a
test dataset. back it up and check atimes before and after.

@Ken: do you know it off the top of your head?

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #668404]: Does duplicity modify atime?

[edso]

Question #668404 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/668404

edso proposed the following answer:
On 30.04.2018 11:22, Tim wrote:
> Tim is still having a problem:
> Yes, atime is a filesystem feature.  But it is possible to
> programatically read and store the original atime, then read the file,
> then restore the original atime — using utimes() — such that atime
> effectively does not change.  I am wondering if Duplicity 'preserves'
> the original atime in such a manner.

- just two more thoughts -

1. 
it looks like the approach above does write atime 2times to the fs, once during 
read, second during restoration of the old atime, which sounds 
counterproductive to your initial desire to cirumvent writing to your SSD.

2.
a quick web search provided
 
https://stackoverflow.com/questions/16464228/python-get-last-reading-time-of-a-file
where it is hinted that _relatime_ only does atime update if the stored atime 
is older than 24hs.

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #659626]: S3 data transfer out - getObject requests

[edso]

Question #659626 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/659626

Status: Open => Answered

edso proposed the following answer:
from the manpage http://duplicity.nongnu.org/duplicity.1.html

"
verify [--compare-data] [--time ] [--file-to-restore ]  


Restore backup contents temporarily file by file and compare against the local 
path’s contents. duplicity will exit with a non-zero error level if any files 
are different. On verbosity level info (4) or higher, a message for each file 
that has changed will be logged. 
The --file-to-restore option restricts verify to that file or folder. The 
--time option allows to select a backup to verify against. The --compare-data 
option enables data comparison (see below).
"

every verify essentially download the whole recent chgain and tries to
restore every file. that should account for theamount of incoming
traffic in your case. ..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #659626]: S3 data transfer out - getObject requests

[edso]

Question #659626 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/659626

Status: Open => Answered

edso proposed the following answer:
On 20.10.2017 14:33, Jasper wrote:
> Question #659626 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/659626
> 
> Status: Answered => Open
> 
> Jasper is still having a problem:
> Right, thanks for the heads up.
> 
> Would it be possible to combine them like this?
> 
> 0 0 * * 1-6 duply system backup
> 0 0 * * 0 duply system backup_verify_and_purge --force
> 
> So a normal backup every day from monday - saturday, and a backup with
> verify and purge on sunday?
> 

looks good to me. if the weekly verify is still to costly you can look
around the net, there are nifty conditional calls, that allow you to
define things like every 2 week or last friday of the month

eg. this one scrubs a btrfs vol of mine, when it's the last friday of
every 3rd month, only if the volume is mounted

0 20 * 3,6,9,12 Fri  mount | grep -q /mnt/raid && ( [ $(date +\%d -d '+7
days') -lt '8' ] && btrfs scrub start /mnt/raid )

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Merge] lp:~dawgfoto/duplicity/fixup1251 into lp:duplicity

[edso]

On 1/21/2018 12:19, Kenneth Loafman wrote:
> Thanks for the catch!  Not sure how it got deleted.
> 

hey,

had a quick look just to find our where it got missing and _couldn't find_ a 
point where it existed earlier. checked
  
http://bazaar.launchpad.net/~duplicity-team/duplicity/0.6-series/view/head:/duplicity/collections.py
  
http://bazaar.launchpad.net/~duplicity-team/duplicity/0.7-series/view/head:/duplicity/collections.py
  
http://bazaar.launchpad.net/~duplicity-team/duplicity/0.8-series/view/head:/duplicity/collections.py
and sampled some earlier revisions in each.

can someone point to a revision of collection.py that held this code in the 
past?

..ede/duply.net

-- 
https://code.launchpad.net/~dawgfoto/duplicity/fixup1251/+merge/336393
Your team duplicity-team is requested to review the proposed merge of 
lp:~dawgfoto/duplicity/fixup1251 into lp:duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Merge] lp:~dawgfoto/duplicity/fixup1251 into lp:duplicity

[edso]

ahhh.. makes sense, thanks for clearing this up!.. ede

On 1/21/2018 13:21, Kenneth Loafman wrote:
> All in series-8 - it was added in rev 1209 to support replicate, then
> removed in 1251 by accident.
> 
> 
> On Sun, Jan 21, 2018 at 5:42 AM, edso <edgar.sol...@web.de> wrote:
> 
>> On 1/21/2018 12:19, Kenneth Loafman wrote:
>>> Thanks for the catch!  Not sure how it got deleted.
>>>
>>
>> hey,
>>
>> had a quick look just to find our where it got missing and _couldn't find_
>> a point where it existed earlier. checked
>>   http://bazaar.launchpad.net/~duplicity-team/duplicity/0.6-
>> series/view/head:/duplicity/collections.py
>>   http://bazaar.launchpad.net/~duplicity-team/duplicity/0.7-
>> series/view/head:/duplicity/collections.py
>>   http://bazaar.launchpad.net/~duplicity-team/duplicity/0.8-
>> series/view/head:/duplicity/collections.py
>> and sampled some earlier revisions in each.
>>
>> can someone point to a revision of collection.py that held this code in
>> the past?
>>
>> ..ede/duply.net
>>
>> --
>> https://code.launchpad.net/~dawgfoto/duplicity/fixup1251/+merge/336393
>> You are subscribed to branch lp:duplicity.
>>
> 


On 1/21/2018 13:21, Kenneth Loafman wrote:
> All in series-8 - it was added in rev 1209 to support replicate, then
> removed in 1251 by accident.
> 
> 
> On Sun, Jan 21, 2018 at 5:42 AM, edso <edgar.sol...@web.de> wrote:
> 
>> On 1/21/2018 12:19, Kenneth Loafman wrote:
>>> Thanks for the catch!  Not sure how it got deleted.
>>>
>>
>> hey,
>>
>> had a quick look just to find our where it got missing and _couldn't find_
>> a point where it existed earlier. checked
>>   http://bazaar.launchpad.net/~duplicity-team/duplicity/0.6-
>> series/view/head:/duplicity/collections.py
>>   http://bazaar.launchpad.net/~duplicity-team/duplicity/0.7-
>> series/view/head:/duplicity/collections.py
>>   http://bazaar.launchpad.net/~duplicity-team/duplicity/0.8-
>> series/view/head:/duplicity/collections.py
>> and sampled some earlier revisions in each.
>>
>> can someone point to a revision of collection.py that held this code in
>> the past?
>>
>> ..ede/duply.net
>>
>> --
>> https://code.launchpad.net/~dawgfoto/duplicity/fixup1251/+merge/336393
>> You are subscribed to branch lp:duplicity.
>>
> 


-- 
https://code.launchpad.net/~dawgfoto/duplicity/fixup1251/+merge/336393
Your team duplicity-team is requested to review the proposed merge of 
lp:~dawgfoto/duplicity/fixup1251 into lp:duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #664091]: Multiple incremental chains on a full backup?

[edso]

Question #664091 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/664091

edso proposed the following answer:
as always,

if you want/need this functionality you are welcome to contribute these
changes. or maybe sponsor the the development by offering a bounty?

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #664503]: cannot import name Gio

[edso]

Question #664503 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/664503

Status: Open => Answered

edso proposed the following answer:
On 2/17/2018 0:02, Dan Jaenecke wrote:
> My OS: Ubuntu 16.04.3 LTS
> Version of duplicity: 0.7.06

hey Dan,

please update to the latest 0.7.16 from tarball or ppa (see
http://duplicity.nongnu.org/). it should show more details on import
errors.

make sure that deja-dup is run with python 2.7.

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #664503]: cannot import name Gio

[edso]

Question #664503 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/664503

Status: Open => Answered

edso proposed the following answer:
Dan,

can you check if the gio files
  https://packages.ubuntu.com/xenial/amd64/python-gi/filelist
exist on your box?

would you be willing to patch your duplicity to have it describe the
error in more detail?

..ede/duply.net


On 2/18/2018 12:33, Dan Jaenecke wrote:
> Question #664503 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/664503
> 
> Status: Answered => Open
> 
> Dan Jaenecke is still having a problem:
> Hi edso,
> 
> same problem with 0.7.16…
> 
> $ PYTHONPATH=/opt/duplicity/lib/python2.7/site-packages 
> /opt/duplicity/bin/duplicity -V
> duplicity 0.7.16
> 
> $ PYTHONPATH=/opt/duplicity/lib/python2.7/site-packages 
> /opt/duplicity/bin/duplicity --verbosity=9 --gio collection-status file:///tmp
> Achtung: Parameter --gio ist veraltet und wird in einer der nächsten 
> Versionen entfernt.
> Bitte verwenden Sie Standarddateinamen.
> Folgendes Archivverzeichnis wird benutzt: 
> /home/daniel/.cache/duplicity/c2731c0788339744944161fd8afb74dd
> Folgender Sicherungsname wird benutzt: c2731c0788339744944161fd8afb74dd
> GPG binary is gpg, version 1.4.20
> Import von duplicity.backends.acdclibackend Succeeded
> Import von duplicity.backends.azurebackend Succeeded
> Import von duplicity.backends.b2backend Succeeded
> Import von duplicity.backends.botobackend Succeeded
> Import von duplicity.backends.cfbackend Succeeded
> Import von duplicity.backends.dpbxbackend Failed: No module named dropbox
> Import von duplicity.backends.gdocsbackend Succeeded
> Import von duplicity.backends.giobackend Succeeded
> Import von duplicity.backends.hsibackend Succeeded
> Import von duplicity.backends.hubicbackend Succeeded
> Import von duplicity.backends.imapbackend Succeeded
> Import von duplicity.backends.lftpbackend Succeeded
> Import von duplicity.backends.localbackend Succeeded
> Import von duplicity.backends.mediafirebackend Succeeded
> Import von duplicity.backends.megabackend Succeeded
> Import von duplicity.backends.multibackend Succeeded
> Import von duplicity.backends.ncftpbackend Succeeded
> Import von duplicity.backends.onedrivebackend Succeeded
> Import von duplicity.backends.par2backend Succeeded
> Import von duplicity.backends.pydrivebackend Succeeded
> Import von duplicity.backends.rsyncbackend Succeeded
> Import von duplicity.backends.ssh_paramiko_backend Succeeded
> Import von duplicity.backends.ssh_pexpect_backend Succeeded
> Import von duplicity.backends.swiftbackend Succeeded
> Import von duplicity.backends.sxbackend Succeeded
> Import von duplicity.backends.tahoebackend Succeeded
> Import von duplicity.backends.webdavbackend Succeeded
> Temporäres Verzeichnis /tmp/duplicity-JQJDoc-tempdir wird benutzt
> Fehlerdetail des Hintergrundprogramms: Traceback (innermost last):
>   File "/opt/duplicity/bin/duplicity", line 1559, in 
> with_tempdir(main)
>   File "/opt/duplicity/bin/duplicity", line 1545, in with_tempdir
> fn()
>   File "/opt/duplicity/bin/duplicity", line 1381, in main
> action = commandline.ProcessCommandLine(sys.argv[1:])
>   File "/opt/duplicity/lib/python2.7/site-packages/duplicity/commandline.py", 
> line 1132, in ProcessCommandLine
> globals.backend = backend.get_backend(args[0])
>   File "/opt/duplicity/lib/python2.7/site-packages/duplicity/backend.py", 
> line 223, in get_backend
> obj = get_backend_object(url_string)
>   File "/opt/duplicity/lib/python2.7/site-packages/duplicity/backend.py", 
> line 211, in get_backend_object
> raise BackendException(_("Could not initialize backend: %s") % 
> str(sys.exc_info()[1]))
>  BackendException: Hintergrundprogramm konnte nicht initialisiert werden: 
> cannot import name Gio
> 
> BackendException: Hintergrundprogramm konnte nicht initialisiert werden:
> cannot import name Gio
>

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #664503]: cannot import name Gio

[edso]

Question #664503 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/664503

edso proposed the following answer:
On 2/18/2018 12:33, Dan Jaenecke wrote:
> $ PYTHONPATH=/opt/duplicity/lib/python2.7/site-packages 
> /opt/duplicity/bin/duplicity -V
> duplicity 0.7.16
> 
> $ PYTHONPATH=/opt/duplicity/lib/python2.7/site-packages 
> /opt/duplicity/bin/duplicity --verbosity=9 --gio collection-status file:///tmp

i see you are running duplicity in shell. in case you are willing to
waive the comfort of deja-dup (which insists on gio) you could use the
standard backends. note the additional given default python libs path
and the missing '--gio'

  PYTHONPATH=/opt/duplicity/lib/python2.7/site-
packages:/usr/lib/python2.7/dist-packages/ /opt/duplicity/bin/duplicity
--verbosity=9 --gio collection-status file:///tmp

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #664503]: cannot import name Gio

[edso]

Question #664503 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/664503

Status: Open => Answered

edso proposed the following answer:
On 2/18/2018 13:12, Dan Jaenecke wrote:
> And yes, I *could* stop using deja-dup, but it is really convenient and I'd 
> prefer to continue using it. For the moment I fixed the problem by having a 
> shell-wrapper for duplicity which discards the --gio-Parameter.

nifty trick ;).. and oh so easy! so are you satisfied by your workaround
or are you willing to dig deeper into why gio fails?

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #670964]: AssertionError in add_filename - backup folder corrupted?

[edso]

Question #670964 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/670964

edso posted a new comment:
hey Kakurady,

just want to reemphasize the necessity of a verify(s) if you really want
to be sure that your restored chain(s) are truely proper!

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Merge] lp:~aaron-whitehouse/duplicity/08-adorn-strings into lp:duplicity

[edso]

question. does it really make sense to adorn singular strings meant as a code 
comment only eg.

 class GlobbingError(Exception):
-"""Something has gone wrong when parsing a glob string"""
+u"""Something has gone wrong when parsing a glob string"""
 pass

..ede

-- 
https://code.launchpad.net/~aaron-whitehouse/duplicity/08-adorn-strings/+merge/350078
Your team duplicity-team is requested to review the proposed merge of 
lp:~aaron-whitehouse/duplicity/08-adorn-strings into lp:duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #670964]: AssertionError in add_filename - backup folder corrupted?

[edso]

Question #670964 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/670964

edso posted a new comment:
Ken,

looks more like

"
AssertionError: ({1: 
'duplicity-inc.20180614T072154Z.to.20180615T072208Z.vol1.difftar.gz'}, 
'duplicity-inc.20180614T072154Z.to.20180615T072208Z.vol1.difftar.gz')
"

this unencrypted compressed file is listed _two_ times bey the file
system because of the corruption and that is why duplicity chokes..
ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #670964]: AssertionError in add_filename - backup folder corrupted?

[edso]

Question #670964 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/670964

Status: Open => Answered

edso proposed the following answer:
sounds exactly like  Bug #1652410 therefor i linked it. should probably
at lest turned into a meaningful error message.

wrt. your problem, i'd suggest
- clone the full drive, to have an untouched copy
- try to repair/chkdsk either one drive
- see if a verify runs through

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #670964]: AssertionError in add_filename - backup folder corrupted?

[edso]

Question #670964 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/670964

Linked to bug: #1652410
https://bugs.launchpad.net/bugs/1652410
"Undescriptive duplicity/collection-status error when the backup directory 
contains two volumes with different file names and same volume number in the 
same backup set"

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Merge] lp:~excitablesnowball/duplicity/s3-onezone-ia into lp:duplicity

[edso]

hey Snow,

nice of you to have added the documentation already in the manpage. +1.. ede
-- 
https://code.launchpad.net/~excitablesnowball/duplicity/s3-onezone-ia/+merge/348941
Your team duplicity-team is requested to review the proposed merge of 
lp:~excitablesnowball/duplicity/s3-onezone-ia into lp:duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Merge] lp:~dawgfoto/duplicity/fixup1252 into lp:duplicity

[edso]

On 24.04.2018 18:20, Martin Nowak wrote:
> What's the corruption scenario that manifest comparison should protect 
> against?
> 
> The manifest contains volume checksums, but those aren't checked against the 
> volumes.

they should, at least during verify/restore

> Eventually the remote manifest is just a copy of the local one. Multiple 
> duplicity instances writing to the same remote should be ruled out as well.

you never know. better safe than sorry.

> 
> The only thing that seems possible is an incomplete backup, but FWIW 
> duplicitly only renames a local temp manifest and uploads it to the remote 
> once the full backup is done.
> 

or some corruption in the backend or local file system or..

what stays is the question how a corrupt manifest would affect an incremental 
or resumed backup?
@Ken: do you want to chime in, as this smells like your area of expertise?  

additionally of course, the question still remains if this check makes sense 
there at all. obviously someone at some point thought so in the past, but no 
harm in revalidating.

..ede/duply.net

-- 
https://code.launchpad.net/~dawgfoto/duplicity/fixup1252/+merge/343816
Your team duplicity-team is requested to review the proposed merge of 
lp:~dawgfoto/duplicity/fixup1252 into lp:duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Merge] lp:~dawgfoto/duplicity/fixup1252 into lp:duplicity

[edso]

hey Martin,

while the patch looks sound, i'd really like to know why the manifest is 
retrieved/decrypted from backend in the first place, before it get's applied.

@Ken?

..ede/duply.net
-- 
https://code.launchpad.net/~dawgfoto/duplicity/fixup1252/+merge/343816
Your team duplicity-team is requested to review the proposed merge of 
lp:~dawgfoto/duplicity/fixup1252 into lp:duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Merge] lp:~dawgfoto/duplicity/fixup1252 into lp:duplicity

[edso]

hey Martin,

are you aware of the "no private key issue/double key approach" as eg. 
described here
  https://lists.gnu.org/archive/html/duplicity-talk/2017-10/msg00015.html
?

On 23.04.2018 15:50, Martin Nowak wrote:
> Seems like it tries to catch corruptions see 
> https://bazaar.launchpad.net/~duplicity-team/duplicity/0.8-series/view/1301/bin/duplicity#L1333
>  and 
> https://bazaar.launchpad.net/~duplicity-team/duplicity/0.8-series/view/1301/duplicity/collections.py#L218.

ok, isee the manifests are compared, which sounds reasonable.

> Given that this is not possible with an encrypted backup (without the key 
> and/or password) 

it is ;) when a private key and passphrase (might be empty) for one of the used 
public keys is available.
as i don't see a way to easily detect a private key available (duply has some 
shell gpg trickery for this though) i don't see a problem logging the gpg error 
as a warning and simply continue.

>
and that the check hasn't been done despite the "non-fatal" error message, it 
seems fine to me to just skip it.
> 

i'd rather log a warning that the comparison was skipped because of the 
decryption impossibility, so the user may be informed that a /surplus/ security 
measure was not applied.

finally: the current mode of allowing incrementals w/o validating the backend 
is error prone[1] and should be replaced in a future version of duplicity. but 
as it stands, nobody took care of it until today.

food for thought ..ede/duply.net

[1] https://bugs.launchpad.net/duplicity/+bug/687295

-- 
https://code.launchpad.net/~dawgfoto/duplicity/fixup1252/+merge/343816
Your team duplicity-team is requested to review the proposed merge of 
lp:~dawgfoto/duplicity/fixup1252 into lp:duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #674847]: Certificate Error when connecting to other-than-Virginia S3 endpoint

[edso]

Question #674847 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/674847

Status: Open => Answered

edso proposed the following answer:
On 08.10.2018 14:37, Eric Christensen wrote:
> New question #674847 on Duplicity:
> https://answers.launchpad.net/duplicity/+question/674847
> 
> I'm trying to use a S3 bucket I created in the Ohio region for my off-site 
> backup storage.  When connecting to the server, however, I get the following 
> error:
> 
> CertificateError: hostname 's3-us-east-2.amazonaws.com.s3.amazonaws.com' 
> doesn't match either of '*.s3.amazonaws.com', 's3.amazonaws.com'
> 
> (That is the address to the Ohio S3 endpoint[0], by the way.)
> 
> My first thought was that Amazon was somehow using a bad wildcard certificate 
> at the endpoint which was causing the problem.  Upon further investigation, 
> it appears that the certificate used is proper[1].  Does Duplicity use a 
> stored certificate for verifying the connection?  Why would this error being 
> happening?
> 
> [0] https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region
> [1] https://www.ssllabs.com/ssltest/analyze.html?d=s3.us-east-2.amazonaws.com
> 

hey Eric,

afaik and according to
  https://en.wikipedia.org/wiki/Wildcard_certificate
"
Limitations
Only a single level of subdomain matching is supported in accordance with RFC 
2818.[7]
"
so the error is valid. where does 's3-us-east-2.amazonaws.com.s3.amazonaws.com' 
come from?
see
  
https://www.ssllabs.com/ssltest/analyze.html?d=s3-us-east-2.amazonaws.com.s3.amazonaws.com

also note from your info above 
  s3-us-east-2.amazonaws.com.s3.amazonaws.com
is not the same as
  s3.us-east-2.amazonaws.com

the aws docs above seem to say 
  s3.us-east-2.amazonaws.com
  s3-us-east-2.amazonaws.com
(prefixed 's3.' or 's3-') are valid [0] .

what is you command line (especially the target url)?

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #674425]: rsync backend Remote filesize 0 does not match local size N, retrying

[edso]

Question #674425 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/674425

edso posted a new comment:
Clemens,

this is fixed in trunk but unreleased so far
  
https://bazaar.launchpad.net/~duplicity-team/duplicity/0.7-series/revision/1365
feel free to manually patch 0.7.18 or use 0.7.17 instead.

@Ken: this is really a showstopper and should be released asap.

..ede/duply.net

On 16.10.2018 16:32, Clemens Fuchslocher wrote:
> Question #674425 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/674425
> 
> Clemens Fuchslocher posted a new comment:
> I also get this error with 0.7.18.1 and the SFTP backend:
> 
> | $ duplicity --version
> | duplicity 0.7.18.1
> |
> | $ duplicity incremental --name "##" --full-if-older-than 15D 
> --gpg-options="--cipher-algo=AES" --num-retries 10 --backend-retry-delay 10 
> --asynchronous-upload --verbosity notice "/srv/##" 
> "par2+sftp://##/;
> | ...
> | Local and Remote metadata are synchronized, no sync needed.
> | Last full backup date: Wed Oct 10 04:00:24 2018
> | Remote filesize 0 for 
> 'duplicity-inc.20181016T020018Z.to.20181016T131506Z.vol1.difftar.gpg' does 
> not match local size 9099547, retrying.
> | Remote filesize 0 for 
> 'duplicity-inc.20181016T020018Z.to.20181016T131506Z.vol1.difftar.gpg' does 
> not match local size 9099547, retrying.
> | Remote filesize 0 for 
> 'duplicity-inc.20181016T020018Z.to.20181016T131506Z.vol1.difftar.gpg' does 
> not match local size 9099547, retrying.
> | Remote filesize 0 for 
> 'duplicity-inc.20181016T020018Z.to.20181016T131506Z.vol1.difftar.gpg' does 
> not match local size 9099547, retrying.
> | Remote filesize 0 for 
> 'duplicity-inc.20181016T020018Z.to.20181016T131506Z.vol1.difftar.gpg' does 
> not match local size 9099547, retrying.
> | Remote filesize 0 for 
> 'duplicity-inc.20181016T020018Z.to.20181016T131506Z.vol1.difftar.gpg' does 
> not match local size 9099547, retrying.
> | Remote filesize 0 for 
> 'duplicity-inc.20181016T020018Z.to.20181016T131506Z.vol1.difftar.gpg' does 
> not match local size 9099547, retrying.
> | Remote filesize 0 for 
> 'duplicity-inc.20181016T020018Z.to.20181016T131506Z.vol1.difftar.gpg' does 
> not match local size 9099547, retrying.
> | Remote filesize 0 for 
> 'duplicity-inc.20181016T020018Z.to.20181016T131506Z.vol1.difftar.gpg' does 
> not match local size 9099547, retrying.
> | Remote filesize 0 for 
> 'duplicity-inc.20181016T020018Z.to.20181016T131506Z.vol1.difftar.gpg' does 
> not match local size 9099547, retrying.
> | --[ Backup Statistics ]--
> | StartTime 1539695707.10 (Tue Oct 16 15:15:07 2018)
> | EndTime 1539695740.43 (Tue Oct 16 15:15:40 2018)
> | ElapsedTime 33.33 (33.33 seconds)
> | SourceFiles 81755
> | SourceFileSize 1926239011 (1.79 GB)
> | NewFiles 44
> | NewFileSize 78586390 (74.9 MB)
> | DeletedFiles 9
> | ChangedFiles 8
> | ChangedFileSize 22545363 (21.5 MB)
> | ChangedDeltaSize 0 (0 bytes)
> | DeltaEntries 61
> | RawDeltaSize 82924326 (79.1 MB)
> | TotalDestinationSizeChange 9099547 (8.68 MB)
> | Errors 0
> | -
> |
> | $ echo $?
> | 0
> 
> The error doesn't happen with 0.7.17.
>

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Merge] lp:~mcuelenaere/duplicity/duplicity into lp:duplicity

[edso]

Review: Approve

nice catch. one wonders what the original author thought. thx! ..ede/duply.net
-- 
https://code.launchpad.net/~mcuelenaere/duplicity/duplicity/+merge/358411
Your team duplicity-team is subscribed to branch lp:duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #674425]: rsync backend Remote filesize 0 does not match local size N, retrying

[edso]

Question #674425 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/674425

Status: Open => Answered

edso proposed the following answer:
On 28.09.2018 00:17, BenRifkah Bergsten-Buret wrote:
> Question #674425 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/674425
> 
> BenRifkah Bergsten-Buret gave more information on the question:
> I did some more digging and this looks to me like a bug in duplicity
> 0.7.18.1. I was able to reproduce it with the both rsync and scp
> backends on the following new Vagrant Box installations:
> 
> ubuntu/bionic64 – built locally. No RPM available
> centos/7 – from EPEL package
> 
> I couldn't reproduce it with duplicity 0.7.17:
> ubuntu/bionic64 – from RPM
> centos/7 – built locally. No RPM available
> 
> I'm happy to provide more details but if this needs to be turned into a
> bug report let me know and I'll provide details there.
> 
> Thanks,
> 

seems to be
"
* Partial fix of bug #1734144 with patch from Joris van Eijden
  - Note: this is really just a partial fix for now, since it only covers
the case where the local size does not yet match the remote size.  A
case where -1 is the returned length is not fixed.
"
  
https://bazaar.launchpad.net/~duplicity-team/duplicity/0.7-series/revision/1354

it does not check if the entry is 'None' which is the case if the backend does 
not support file size meta data fetching (most do not) 
  
https://bazaar.launchpad.net/~duplicity-team/duplicity/0.8-series/view/head:/duplicity/backend.py#L616

for now you can stick w/ 0.7.17.. hopefully Ken will fix this soonish,
as it probably breaks lot's of installations.

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #674425]: rsync backend Remote filesize 0 does not match local size N, retrying

[edso]

Question #674425 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/674425

Status: Open => Answered

edso proposed the following answer:
On 26.09.2018 02:07, BenRifkah Bergsten-Buret wrote:
> New question #674425 on Duplicity:
> https://answers.launchpad.net/duplicity/+question/674425
> 
> I'm trying to get backups running from my CentOS server to a RHEL remote 
> server. I keep getting this error when I try to back up:
> 
> "Remote filesize 0 for 'duplicity-full.20180925T231400Z.vol1.difftar.gz' does 
> not match local size 496, retrying."
> 
> It appears  5 times and then duplicity gives up. When I check the files on 
> the remote server they have a non-zero size. It looks like the rsync backend 
> is having trouble checking and reporting the size. I have a minimum example 
> that produces the message but syncs to a different directory on the source 
> host.
> 
> Here's the info level log output:
> 
> "$ duplicity full --no-encryption pkgs 
> rsync://localhost//home/CHSCC/bburnett/duplicity_backup_test/pkgs/
> Local and Remote metadata are synchronized, no sync needed.
> Last full backup date: none
> Remote filesize 0 for 'duplicity-full.20180926T000202Z.vol1.difftar.gz' does 
> not match local size 496, retrying.
> Remote filesize 0 for 'duplicity-full.20180926T000202Z.vol1.difftar.gz' does 
> not match local size 496, retrying.
> Remote filesize 0 for 'duplicity-full.20180926T000202Z.vol1.difftar.gz' does 
> not match local size 496, retrying.
> Remote filesize 0 for 'duplicity-full.20180926T000202Z.vol1.difftar.gz' does 
> not match local size 496, retrying.
> Remote filesize 0 for 'duplicity-full.20180926T000202Z.vol1.difftar.gz' does 
> not match local size 496, retrying.
> --[ Backup Statistics ]--
> StartTime 1537920123.57 (Tue Sep 25 17:02:03 2018)
> EndTime 1537920123.61 (Tue Sep 25 17:02:03 2018)
> ElapsedTime 0.03 (0.03 seconds)
> SourceFiles 29
> SourceFileSize 86 (86 bytes)
> NewFiles 29
> NewFileSize 86 (86 bytes)
> DeletedFiles 0
> ChangedFiles 0
> ChangedFileSize 0 (0 bytes)
> ChangedDeltaSize 0 (0 bytes)
> DeltaEntries 29
> RawDeltaSize 0 (0 bytes)
> TotalDestinationSizeChange 496 (496 bytes)
> Errors 0
> -
> 
> bburnett@chrusm0:~$ ssh localhost ls -lG duplicity_backup_test/pkgs/ 
> 2>/dev/null total 5
> -rw---. 1 bburnett 193 Sep 25 17:02 
> duplicity-full.20180926T000202Z.manifest
> -rw---. 1 bburnett 496 Sep 25 17:02 
> duplicity-full.20180926T000202Z.vol1.difftar.gz
> -rw---. 1 bburnett 509 Sep 25 17:02 
> duplicity-full-signatures.20180926T000202Z.sigtar.gz
> $ duplicity --version
> duplicity 0.7.18.1
> $ lsb_release -a
> LSB Version:
> :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:lan
> guages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch
> Distributor ID: CentOS
> Description:CentOS Linux release 7.5.1804 (Core)
> Release:7.5.1804
> Codename:   Core"
> 
> (SSH Auth is being handled by an agent)
> 
> I don't have this problem when I try the same rsync-to-localhost using Ubuntu 
> on Windows 10 with duplicity 0.7.17. Has anyone seen this before?
> 

hey Ben,

nope, at least this never came up on the list until now. rsync backend
does not support reporting file sizes, so this error should never occur.

1.
what is your duplicity version?

2.
please run the backup with max. verbosity '-v9' and put the _full_ terminal 
output on a filehoster/pastebin or such and post the link. somewhere 
view/downloadable. obfuscate private strings in it beforehand if needed.

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #677428]: "Restore" gives the following:Failed to read /tmp/duplicity-Rtd2wO-tempdir/mktemp-67n08q-1. Can I recover any of my data?

[edso]

Question #677428 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/677428

Status: Open => Answered

edso proposed the following answer:
On 22.01.2019 04:57, trevj wrote:
> Question #677428 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/677428
>
> Status: Needs information => Open
>
> trevj gave more information on the question:
>
> On 20/1/19 3:12 am, Kenneth Loafman wrote:
>> Your question #677428 on Duplicity changed:
>> https://answers.launchpad.net/duplicity/+question/677428 Status: Open
>> => Needs information Kenneth Loafman requested more information: Some
>> questions, - Are you running the same user as duplicity when testing
>> gzip? - Is your system up-to-date? - Is /media/trevorj a Linux
>> filesystem? Just shooting in the dark here.
>
>
> On 20/1/19 3:12 am, Kenneth Loafman wrote:
>>
>> Your question #677428 on Duplicity changed:
>> https://answers.launchpad.net/duplicity/+question/677428 Status: Open
>> => Needs information Kenneth Loafman requested more information: Some
>> questions, - Are you running the same user as duplicity when testing
>> gzip? Yes
>
>>
>> - Is your system up-to-date? Yes - Is /media/trevorj a Linux
>> filesystem? Yes
>
>>
>> Just shooting in the dark here.
> Desperate times call for desperate actions:
>
> 1    I copied all the files on backup to a temp directory on my main
> hard disk.
>
> 2    In the temp directory I ran
>
>      for t in duplicity-full.20181208T043900Z.*.difftar.gz; do tar xf
> $t; done
>
> that's just the files in the first back up on the disk (Dec 8th 2018)
>
> 3    That created two directories multivol_snapshot and snapshot.
> snapshot was mainly empty folders but multivol_snapshot held most of my
> data. Unfortunately. the files were in slices numbered 1, 2 3 etc. 
> "cat" worked to join test files together. But I cannot imagine going
> through all the folders "cat" ing all the files.
>
>
> I found this script
>
> |find multivol_snapshot/ -type f -printf '%h\0' | \| |||sort -uz | \|
> |||xargs -0 -n 1 sh -c 'cd "$1" ; cat $(ls | sort -n) > content' spacer |
> It lists the containing directory for every file, removes duplicates
> from the list, then goes to each directory and creates a |content |file
> from the fragments there. (|spacer |is just to make|$1 |work.)
>
>
> But it's still messy.
>

Trevj,

if you can manually extract _all_ files then duplicity should as well.
btw. the "official" disaster recovery document resides here
https://wiki.gnome.org/Apps/DejaDup/Help/Restore/WorstCase#Restoring_by_Hand
but you seem to have found it already.

1.
first of all, can you show a 'df -h' that show there is ample space in your 
/tmp folder?

2.
run duplicity w/ the backup data copied from another file system (just to rule 
out read errors)

3.
if the error persists run it w/ max. verbosity (-v9) and '--ignore-errors' and 
redirect the output to a log file. here's the description from the man page
"
Try to ignore certain errors if they happen. This option is only intended to 
allow the restoration of a backup in the face of certain problems that would 
otherwise cause the backup to fail. It is not ever recommended to use this 
option unless you have a situation where you are trying to restore from backup 
and it is failing because of an issue which you want duplicity to ignore. Even 
then, depending on the issue, this option may not have an effect.
Please note that while ignored errors will be logged, there will be no summary 
at the end of the operation to tell you what was ignored, if anything. If this 
is used for emergency restoration of data, it is recommended that you run the 
backup in such a way that you can revisit the backup log (look for lines 
containing the string IGNORED_ERROR).

If you ever have to use this option for reasons that are not understood or 
understood but not your own responsibility, please contact duplicity 
maintainers. The need to use this option under production circumstances would 
normally be considered a bug.
"

might be that just one volume is corrupted.

4.
possible but unlikely. we had a bug years ago where '--no-compression' didn't 
work.
  https://serverfault.com/questions/726525/cannot-restore-duplicity-backup
but your case looks different.

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #677428]: "Restore" gives the following:Failed to read /tmp/duplicity-Rtd2wO-tempdir/mktemp-67n08q-1. Can I recover any of my data?

[edso]

Question #677428 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/677428

Status: Open => Answered

edso proposed the following answer:
Trevorj,

as '--ignore-errors' did not help. let's try something else.

1.
can gunzip all files and capture the output '> log.txt 2>&1' (notice the STDERR 
redirect) ? send it so i can see if there are errors.

2.
after that you can try to restore on the resulting files using 
'--no-compression', although duplicity should detect it as uncompressed backup 
by itself.

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #676836]: Can I Encrypt directly with a passphrase? (not using Public/Private keys)

[edso]

Question #676836 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/676836

Status: Open => Answered

edso proposed the following answer:
symmetric encryption is already the default, if no key is given.

fronm the man page section env vars
"
PASSPHRASE
This passphrase is passed to GnuPG. If this is not set, the user will be 
prompted for the passphrase.
"
http://duplicity.nongnu.org/duplicity.1.html#sect6

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #676995]: Threading with Backblaze B2?

[edso]

Question #676995 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/676995

edso proposed the following answer:
and it's not only the backend infrastructure.

currently backup file creation (volumes, manifest ...) is also running
in the same thread, meaning a volume is created and uploaded and only
after that the next is created etc.. exception being the
'--asynchronous-upload' where the next volume is prepared during the
upload of the previous volume already, but only during upload and
limited to one.

to conclude, duplicity currently lacks a threading architecture that
utilizes multiple cores or optimizes bandwidth.

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Question #677428]: "Restore" gives the following:Failed to read /tmp/duplicity-Rtd2wO-tempdir/mktemp-67n08q-1. Can I recover any of my data?

[edso]

Question #677428 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/677428

Status: Open => Answered

edso proposed the following answer:
On 10.01.2019 07:28, trevj wrote:
> New question #677428 on Duplicity:
> https://answers.launchpad.net/duplicity/+question/677428
> 
> I backed-up my home folder with Ubuntu "Backup" prior to re-installing 
> Ubuntu18.04.1. After installation, attempting to restore my data gave the 
> following:
> 
> Failed to read /tmp/duplicity-Rtd2wO-tempdir/mktemp-67n08q-1: ( 'exceptions.IOError'>, IOError('Not a gzipped file',),  0x7f7dae2aa680>)
> 
> I have 4 timed backups on the back-up disk and all gave the same error. Is 
> there any way that I could restore at least some of my folder, or have I 
> destroyed my system?
> 

trevj,

please provide

1.
the full command you use to run duplicity

2.
what is your duplicity version?

3.
ideally run duplicity with max. verbosity '-v9' and provide the complete 
resulting terminal output. maybe by redirecting it to file 'command > 
/tmp/out.txt 2>1'
and provide it via pastebin or some download service.

..ede/duply.net

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Merge] lp:~okrasz/duplicity/duplicity into lp:duplicity

[edso]

Review: Approve

looks good to me.. ede/duply.net
-- 
https://code.launchpad.net/~okrasz/duplicity/duplicity/+merge/361352
Your team duplicity-team is subscribed to branch lp:duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp

Re: [Duplicity-team] [Merge] lp:~okrasz/duplicity/duplicity into lp:duplicity

[edso]

Review: Needs Fixing

hi,

looks good overall. small nitbits though..

1.
please insert the manpage entry where it is alphabetically correct.

2.
manpage text. "storring" can be removed -> 'Standard storage tier used for 
backup files (Hot|Cool|Archive).' 

thanks!.. ede/duply.net
-- 
https://code.launchpad.net/~okrasz/duplicity/duplicity/+merge/361352
Your team duplicity-team is subscribed to branch lp:duplicity.

___
Mailing list: https://launchpad.net/~duplicity-team
Post to : duplicity-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~duplicity-team
More help   : https://help.launchpad.net/ListHelp