Re: [BackupPC-users] BPC 4 very slow

[Les Mikesell]

On Wed, Feb 10, 2016 at 12:31 PM, Gandalf Corvotempesta
 wrote:
> 2016-02-10 16:34 GMT+01:00 Les Mikesell :
>> If you want to keep more than one full shouldn't FullKeepCnt and maybe
>> FullKeepMin be higher?
>
> As wrote many, many, many, many times before: IT DOESN'T WORK.
> I'm trying any possible combination with settings trying to get at
> least 1 backup per day
> but is impossible, every time BPC starts a new full backup (and is
> running a full every time),
> the older one is removed, thus i'm loosing days.

Sorry, but you did post something with Full min = 1 in the last
message complaining about not keeping more than one.  I don't
understand it either, but I don't think you are going to find the
answer with combinations that shouldn't be expected to work.  I'll
second the recommendation to set up a tiny share on a fast cycle so
you can experiment without wasting a lot of time or space.

-- 
   Les Mikesell
 lesmikes...@gmail.com

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Les Mikesell]

On Wed, Feb 10, 2016 at 4:40 AM, Gandalf Corvotempesta
 wrote:
> 2016-02-07 17:25 GMT+01:00 Les Mikesell :
>> How many filled backups are you configured to keep?  And have you
>> adjusted the FillCycle value accordingly?  According to the docs, if
>> there aren't any filled backups other than the most recent, then
>> FillKeepPeriod won't have any effect.   It looks like if you adjust
>> the Full/incremental schedules out of sync with default that gives v3
>> behavior you have to tweak the FillCycle to create the filled copies
>> you want to keep.
>
> I've tried in any possible combinations. It doesn't work. When removing a 
> filled
> backup, the whole backup is removed and this is *TOTALLY WRONG* bacause
> we loose recovery points.
>
> actually i'm trying with FillCycle = 7, Full min = 1, full frequency 6.97
>
> Let's see, but biggest issue is the backup removal.
>

If you want to keep more than one full shouldn't FullKeepCnt and maybe
FullKeepMin be higher?

-- 
  Les Mikesell
lesmikes...@gmail.com

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-02-10 16:34 GMT+01:00 Les Mikesell :
> If you want to keep more than one full shouldn't FullKeepCnt and maybe
> FullKeepMin be higher?

As wrote many, many, many, many times before: IT DOESN'T WORK.
I'm trying any possible combination with settings trying to get at
least 1 backup per day
but is impossible, every time BPC starts a new full backup (and is
running a full every time),
the older one is removed, thus i'm loosing days.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Stephen]

On Wed, 10 Feb 2016, Gandalf Corvotempesta wrote:

> As wrote many, many, many, many times before: IT DOESN'T WORK.

I don't think shouting will enamor anyone for you or your plight. Neither 
will repeating the same complaint ad infinitum.

> I'm trying any possible combination with settings trying to get at
> least 1 backup per day
> but is impossible, every time BPC starts a new full backup (and is
> running a full every time),
> the older one is removed, thus i'm loosing days.

I'm reminded of ESR's "How To Ask Questions The Smart Way," here.


It's apparent that no one else that's using bpc v4 seems to be having this 
problem. Nevertheless, it's alpha code; it's possible you've found an 
edge-case or other bug. Congratulations!

Might I suggest you do some diagnosis? You're more likely to receive help 
if you can at least report the *exact* conditions which lead to the bug 
(and the conditions that don't). Better, if you add some debugging and find 
the bug, report those details. Best, fix it if your coding talents allow.

I wish I had time to replicate your problem and locate your bug. But I 
don't. However if I were in your shoes, I'd probably begin by downloading a 
copy of the code and checking it into my favorite revision control system. 
Then install a server and one or more client instances with some easily 
trackable, changing data -- maybe a couple of cron jobs to 
create/append/delete a few tiny files every few minutes. These machines 
could be VMs, possibly docker instances, etc. Whatever you're familiar with 
and works for you.

Then configure a very short backup period for the clients, an hourly wake 
interval with no black periods, etc. and observe. Change variables *one at 
a time* to see what behavior changes. Add some debugging to the BackupPC 
code at likely critical points. Lather, rinse, repeat. Periodically check 
your changes back into your git/subversion/whatever repo so you can 
backtrack and/or compare if needed.

I'll wager that you'll have a better understanding of the relevant issues 
with a total investment of only 2-4 hours of your time (maybe 24-48 hours 
of wall time).

Just my $0.02.

Best of luck,
Stephen

> --
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140
> ___
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
>

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-02-07 17:25 GMT+01:00 Les Mikesell :
> How many filled backups are you configured to keep?  And have you
> adjusted the FillCycle value accordingly?  According to the docs, if
> there aren't any filled backups other than the most recent, then
> FillKeepPeriod won't have any effect.   It looks like if you adjust
> the Full/incremental schedules out of sync with default that gives v3
> behavior you have to tweak the FillCycle to create the filled copies
> you want to keep.

I've tried in any possible combinations. It doesn't work. When removing a filled
backup, the whole backup is removed and this is *TOTALLY WRONG* bacause
we loose recovery points.

actually i'm trying with FillCycle = 7, Full min = 1, full frequency 6.97

Let's see, but biggest issue is the backup removal.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-31 10:01 GMT+01:00 Gandalf Corvotempesta
:
> I have some updates.
> Now seems to working properly, no more deleted backups. I have #1, #3,
> #4 .. #8
> So, the only missing backups are #0 and #2
> #1 and #3 are both incremental unfilled, #4 is a filled full and from
> #5 to #7 are incremental unfilled.
> #8 (today) is incremental unfilled as it's the last backup.

Still doesn't work.
Every time BPC is removing a filled backup, the *whole* backup is
removed, thus i'm still having
many unavailable backup points. For example, now i'm missing #0, #2,
#4, #8, #12, #15

As i'm backing up once a day, having 6 unavailable backups meas that I
have 6 days unavailable for restore.

I've tried to change full backup mins and full backup counts in almost
every possible combinations, nothing change,
simply, it doesn't work.

What I' would like to archieve is 1 full every 15 days, 1 incremental
every other days, keeping 2 fulls and 30 incrementals
for example:

1 full on February 1, many incrementals from February 2 to February
14, 1 full on February 15, many incrementals from February 16 up to
February 29

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-27 20:56 GMT+01:00 Gandalf Corvotempesta
:
> Actually is removing the WHOLE backup, making some days unavailable to 
> restore.

I have some updates.
Now seems to working properly, no more deleted backups. I have #1, #3,
#4 .. #8
So, the only missing backups are #0 and #2
#1 and #3 are both incremental unfilled, #4 is a filled full and from
#5 to #7 are incremental unfilled.
#8 (today) is incremental unfilled as it's the last backup.

Thhis seems to be correct, right?

2 things are still not working:

1) fsck is still run for every backups, after every backups. Not only
after a failure like I thought before
2) graphs in home are not generated properly. I have rrdtools
installed and BPC is running it, but not graph are plotted.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Bowie Bailey]

On 1/28/2016 9:32 AM, Gandalf Corvotempesta wrote:
> 2016-01-27 23:24 GMT+01:00 Gandalf Corvotempesta
> :
>> I had this:
>>
>> $Conf{FullPeriod} = 27.97;
>>
>> now changed to
>>
>> $Conf{FullPeriod} = 7.97;
>>
>> just in case bpc was parsing only the first digit.
> I've also noticed that original configuration is using float values,
> but when saving from webinterfaces, all is converted to strings:
>
> Config saved by web interface:
> # grep '$Conf{FullPeriod}' config.pl
> $Conf{FullPeriod} = '6.97';
>
> Original config, prior saving from web.
> # grep '$Conf{FullPeriod}' config.pl.old
> $Conf{FullPeriod} = 7.97;
>
>
> In the first case, FullPeriod is a string, in the second, is a float.
> Now i'm trying with the "string" version. let's see

This is Perl.  There is no real distinction between strings, integers, 
and floats.  7.97, '7.97', and "7.97" (along with a few other more 
obscure variants) will all be interpreted the same.

-- 
Bowie

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Les Mikesell]

On Thu, Jan 28, 2016 at 9:03 AM, Bowie Bailey  wrote:
> On 1/28/2016 9:32 AM, Gandalf Corvotempesta wrote:
>>>
>> Config saved by web interface:
>> # grep '$Conf{FullPeriod}' config.pl
>> $Conf{FullPeriod} = '6.97';
>>
>> Original config, prior saving from web.
>> # grep '$Conf{FullPeriod}' config.pl.old
>> $Conf{FullPeriod} = 7.97;
>>
>>
>> In the first case, FullPeriod is a string, in the second, is a float.
>> Now i'm trying with the "string" version. let's see
>
> This is Perl.  There is no real distinction between strings, integers,
> and floats.  7.97, '7.97', and "7.97" (along with a few other more
> obscure variants) will all be interpreted the same.
>

Yes, perl treats variables as all possible types simultaneously so you
can use them without conversions - but on the other hand there is
difference between '6.97' and 7.97.   If that isn't a typo, something
odd happened.

-- 
  Les Mikesell
 lesmikes...@gmail.com

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-28 17:42 GMT+01:00 Les Mikesell :
> on the other hand there is
> difference between '6.97' and 7.97.   If that isn't a typo, something
> odd happened.

That's ok, i've changed the config through admin panel to change
FullPeriod from 7.97 to 6.97 that's why I saw the numbers saved as
strings.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-27 23:24 GMT+01:00 Gandalf Corvotempesta
:
> I had this:
>
> $Conf{FullPeriod} = 27.97;
>
> now changed to
>
> $Conf{FullPeriod} = 7.97;
>
> just in case bpc was parsing only the first digit.

I've also noticed that original configuration is using float values,
but when saving from webinterfaces, all is converted to strings:

Config saved by web interface:
# grep '$Conf{FullPeriod}' config.pl
$Conf{FullPeriod} = '6.97';

Original config, prior saving from web.
# grep '$Conf{FullPeriod}' config.pl.old
$Conf{FullPeriod} = 7.97;


In the first case, FullPeriod is a string, in the second, is a float.
Now i'm trying with the "string" version. let's see

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-28 16:03 GMT+01:00 Bowie Bailey :
> This is Perl.  There is no real distinction between strings, integers,
> and floats.  7.97, '7.97', and "7.97" (along with a few other more
> obscure variants) will all be interpreted the same.

Thanks for clarification.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-27 22:50 GMT+01:00 Les Mikesell :
> And yet, other people who haven't made your changes don't see that
> issue.  I don't know why it would happen but I wonder if your
> FullPeriod is somehow getting parsed as 2.

I had this:

$Conf{FullPeriod} = 27.97;

now changed to

$Conf{FullPeriod} = 7.97;

just in case bpc was parsing only the first digit.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-26 22:58 GMT+01:00 Gandalf Corvotempesta
:
> Let's see what happens this night.

Still another full is running. (obviously, there are no full in the
pool, as BPC is removing previous backups on every run)

It makes a full, then the day after it makes an incremental and
removed the previous full. After that, the next backup would
be another full (as there are no more full available). And so on

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Holger Parplies]

Gandalf Corvotempesta wrote on 2016-01-27 11:28:24 +0100 [Re: [BackupPC-users] 
BPC 4 very slow]:
> [basically the same thing over and over again]

In case it's not obvious, adjust the settings so they specify to do what you
want. Before that, preferably, understand the settings. Use the suitable tool
for your purpose. You keep repeating that you get no advantage from pooling,
because all your files constantly change, and thus you must keep only minimal
history. But you want to restore things you are not keeping. Strange.

Hope that helps.

Regards,
Holger

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-27 20:19 GMT+01:00 Kris Lou :
> Your $Conf{FullKeepCnt} = 1, which means to keep 1 filled backup. (Not
> necessarily 1 Full backup)

That's ok for me. Probably i'll increase that to 2, but even 1 would be ok

> After a successful backup (full or incr), the most recent is your 1 filled
> backup, and so it removes the oldest filled which happens to be your
> previous Full.  Then it'll try to take a new Full on the next run.

That's wrong.
Ok to remove the "fill" from older backups (according to my
FullKeepCount setting)
but AFAIK, this should remove common files between the filled backup
and the previous one.

A sort of reverse incremental, where the last backup is always a full
copy and the previous
backup is a differential backup storing only different files

Actually is removing the WHOLE backup, making some days unavailable to restore.

> You might have stumbled upon an edge case where FullKeepCnt <
> FullKeepCntMin.  And I also wonder about the integrity of the incr backups
> where the filled source of reverse deltas has been removed (i.e. #1, based
> upon #2, but then #2 was removed after successful completion of #3).

Exaclty what i'm trying to understand. If the filled backup is missing, probably
al subsequent unfilled backup would be inconsistent.

> But yeah, experiment with FullKeepCnt?  If you want to take 1 full every 30
> days, then FullKeepCnt = 2 ... I would think.

Already tried with no luck. BPC is still deleting the whole backup.

> At the very least, thanks for bringing some more v4 attention to the list.

v4 is a great software, but actually, is impossible to use.
Probably there is something wrong in my configuration, but i've posted
all customized parameters (mostly the retention levels) to the list
and noone told me about possible issues.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Kris Lou]

I've spent entirely too much time reading the v4 docs, but here's how I
interpret it (and take with a grain of salt as I don't run v4):

Your $Conf{FullKeepCnt} = 1, which means to keep 1 filled backup. (Not
necessarily 1 Full backup)

After a successful backup (full or incr), the most recent is your 1 filled
backup, and so it removes the oldest filled which happens to be your
previous Full.  Then it'll try to take a new Full on the next run.

You might have stumbled upon an edge case where FullKeepCnt <
FullKeepCntMin.  And I also wonder about the integrity of the incr backups
where the filled source of reverse deltas has been removed (i.e. #1, based
upon #2, but then #2 was removed after successful completion of #3).

But yeah, experiment with FullKeepCnt?  If you want to take 1 full every 30
days, then FullKeepCnt = 2 ... I would think.

At the very least, thanks for bringing some more v4 attention to the list.

-Kris




Kris Lou
k...@themusiclink.net

On Wed, Jan 27, 2016 at 9:04 AM, Holger Parplies <wb...@parplies.de> wrote:

> Gandalf Corvotempesta wrote on 2016-01-27 11:28:24 +0100 [Re:
> [BackupPC-users] BPC 4 very slow]:
> > [basically the same thing over and over again]
>
> In case it's not obvious, adjust the settings so they specify to do what
> you
> want. Before that, preferably, understand the settings. Use the suitable
> tool
> for your purpose. You keep repeating that you get no advantage from
> pooling,
> because all your files constantly change, and thus you must keep only
> minimal
> history. But you want to restore things you are not keeping. Strange.
>
> Hope that helps.
>
> Regards,
> Holger
>
>
> --
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
> ___
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
>
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-27 21:32 GMT+01:00 Norm Legare :
> I think he meant use v4 as its configured "out-of-the-box".

I'm using it "out-of-the-box" except for incremental keep count and
mail notification (and exclude dirs for rsync)
I've not customized anything that could lead to issue to backup removal.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Les Mikesell]

On Wed, Jan 27, 2016 at 3:25 PM, Gandalf Corvotempesta
 wrote:
> 2016-01-27 21:32 GMT+01:00 Norm Legare :
>> I think he meant use v4 as its configured "out-of-the-box".
>
> I've not customized anything that could lead to issue to backup removal.
>

And yet, other people who haven't made your changes don't see that
issue.  I don't know why it would happen but I wonder if your
FullPeriod is somehow getting parsed as 2.

-- 
  Les Mikesell
lesmikes...@gmail.com

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-27 9:19 GMT+01:00 Gandalf Corvotempesta
:
> It makes a full, then the day after it makes an incremental and
> removed the previous full. After that, the next backup would
> be another full (as there are no more full available). And so on

This is what I mean, look at the screenshot.

Our log is as follow:

2016-01-23 09:38:07 full backup 0 complete, 2912565 files, 2912565
bytes, 1 xferErrs (0 bad files, 0 bad shares, 1 other)
2016-01-24 01:59:46 incr backup 1 complete, 2905889 files, 2905889
bytes, 0 xferErrs (0 bad files, 0 bad shares, 0 other)
2016-01-25 06:28:16 full backup 2 complete, 2915983 files, 2915983
bytes, 13 xferErrs (0 bad files, 0 bad shares, 13 other)
2016-01-26 16:12:06 incr backup 3 complete, 2927190 files, 2927190
bytes, 0 xferErrs (0 bad files, 0 bad shares, 0 other)
2016-01-27 09:25:01 full backup 4 complete, 2912699 files, 2912699
bytes, 9 xferErrs (0 bad files, 0 bad shares, 9 other)

BPC did a full on 2016-01-22 (#0), 2016-01-25 (#2), 2016-01-27 (#4)

After each backup, it removes the older "filled" backup. In my case,
after each incremental (#1, #3) filled backups #0 and #2 was remved.
2 hours ago, a new full was made. This night probably it will do an
incremental, and as think, it will remove the today full.

This is absolutely wrong because i'm missing restore points. I'm
unable to restore from 2016-01-23, 2016-01-25 and tomorrow i will
unable to restore from 2016-01-27

AFAIK, the previous backup must not be totally removed, but just
converted from "filled" to "unfilled" by removing just *common* files
between last and previous one.
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-25 18:22 GMT+01:00 Gandalf Corvotempesta
:
> Let's see if tonight backup #0 would be removed as usual

As expected, srv3 is now in "delete #0" phase.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Stephen]

On Tue, 26 Jan 2016, Gandalf Corvotempesta wrote:

> 2016-01-26 16:46 GMT+01:00 Gandalf Corvotempesta
> :
>> As expected, srv3 is now in "delete #0" phase.
>
> I think that BPC is trying to remove "common" files between the last
> executed backup and the previous one
> because filled is only the last.
>
> So, in case of #1, it tries to remove common files from #0, but for
> some strange reason, is removing all files and the whole backup.
>
> The same happens for every backup. With #3, it remove the whole #2.

Have you seen this in the v4 docs?:


$Conf{FullKeepCnt} = 1;

 Number of filled backups to keep. Must be >= 1.

 Note: Starting in V4+, deleting backups is done based on Fill/Unfilled, 
not whether the original backup was full/incremental. reasons these 
parameters continue to be called FullKeepCnt, rather than FilledKeepCnt. If 
$Conf{FillCycle} is 0, then full backups continue to be filled, so the 
terms are interchangeable. For V3 backups, the expiry settings have their 
original meanings.

 In the steady state, each time a full backup completes successfully the 
oldest one is removed. If this number is decreased, the extra old backups 
will be removed.


Re-read the last paragraph.

A similar admonishment applies to IncrKeepCnt:


Note: Starting in V4+, deleting backups is done based on Fill/Unfilled, not 
whether the original backup was full/incremental. For historical reasons 
these parameters continue to be called IncrKeepCnt, rather than 
UnfilledKeepCnt. If $Conf{FillCycle} is 0, then incremental backups 
continue to be unfilled, so the terms are interchangeable. For V3 backups, 
the expiry settings have their original meanings.

In the steady state, each time an incr backup completes successfully the 
oldest one is removed. If this number is decreased, the extra old backups 
will be removed.


Now, all of that said... I'd suggest you change your FullKeepCnt value to 2 
(or greater) and see what happens; there may be a hidden edge-case there. I 
have not seen the problem you're having. My "Schedule" settings are:

FullPeriod: 29.5
FillCycle: 0
FullKeepCnt: 5, 0, 6, 0, 2, 0, 1
FullKeepCntMin: 1
FullAgeMax: 90
IncrPeriod: 0.5
IncrKeepCnt: 30
IncrKeepCntMin: 1
IncrAgeMax: 30

You may also want to read the FillCycle setting documentation. It can 
affect which backups are filled/unfilled, but I'd recommend changing only 
one variable at a time.

Hope this helps!

Cheers,
Stephen


--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-26 21:13 GMT+01:00 Stephen :
>  In the steady state, each time a full backup completes successfully the
> oldest one is removed. If this number is decreased, the extra old backups
> will be removed.

So, is normal that BPC is totally removing the older backup after a
successful one ?
Let's see what happens this night.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-26 22:58 GMT+01:00 Gandalf Corvotempesta
:
> So, is normal that BPC is totally removing the older backup after a
> successful one ?
> Let's see what happens this night.

Anyway, this is totally wrong as i'm unable to restore files from an
older backup.
For example, I need 30 backups and at least 2 full
If BPC remove the oldest one, i'll loose one day of backup.

Actually I have:
#1: 1/23 23:43
#3: 1/25 23:40

as you can see, this is wrong.
I also need #0 (1/22) and #2 (1/24)

I've missed two days of restores. What If i need to restore a file as
it was on 1/24/2016 ? I can't because BPC has removed the whole
backup.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-24 23:17 GMT+01:00 Adam Goryachev :
> Could you please try changing the share name to something other than full.

I've changed the share name to "everything" and added a brand new
client to backup.

2016-01-25 09:49:44 full backup started for directory everything
2016-01-25 18:06:03 full backup 0 complete, 2275732 files, 2275732
bytes, 1 xferErrs (0 bad files, 0 bad shares, 1 other)

Let's see if tonight backup #0 would be removed as usual

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-24 23:17 GMT+01:00 Adam Goryachev :
> Could you please try changing the share name to something other than full.

Changed share name to "everything".
Now I'm running a brand new backup to a brand new client.

Let's see.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Les Mikesell]

On Mon, Jan 25, 2016 at 11:22 AM, Gandalf Corvotempesta
 wrote:
> 2016-01-24 23:17 GMT+01:00 Adam Goryachev 
> :
>> Could you please try changing the share name to something other than full.
>
> I've changed the share name to "everything" and added a brand new
> client to backup.
>
> 2016-01-25 09:49:44 full backup started for directory everything
> 2016-01-25 18:06:03 full backup 0 complete, 2275732 files, 2275732
> bytes, 1 xferErrs (0 bad files, 0 bad shares, 1 other)
>
> Let's see if tonight backup #0 would be removed as usual

It might be worth looking through the web interface at the values for
all the settings.  Sometimes you can have a syntax error in the text
file that is just interpreted by perl as something unexpected -
something like using double quotes instead of single around a string
containing the @ symbol.   The web editor should show you how perl
parses the settings.

-- 
Les Mikesell
lesmikes...@gmail.com

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-23 12:10 GMT+01:00 Gandalf Corvotempesta
:
> Let's see what would happen this night.

The same happened again.

2016-01-22 22:41:19 Created directory /var/backups/backuppc/pc/srv1/refCnt
2016-01-22 22:41:19 full backup started for directory full
2016-01-23 11:59:06 full backup 0 complete, 4628296 files, 4628296
bytes, 1 xferErrs (0 bad files, 0 bad shares, 1 other)
2016-01-23 23:43:08 incr backup started for directory full
2016-01-24 01:21:25 incr backup 1 complete, 4630086 files, 4630086
bytes, 1 xferErrs (0 bad files, 0 bad shares, 1 other)

Now #0 is missing. I only have #1 set as incremental.

The same for srv2


2016-01-23 12:00:01 Running 2 BackupPC_nightly jobs from 0..15 (out of 0..15)
2016-01-23 12:00:01 Running BackupPC_nightly -m -P 6 0 127 (pid=4993)
2016-01-23 12:00:01 Running BackupPC_nightly -P 6 128 255 (pid=4994)
2016-01-23 12:00:01 Next wakeup is 2016-01-23 23:00:00
2016-01-23 12:00:01 BackupPC_nightly now running
BackupPC_refCountUpdate -m -s -c -P 6 -r 128-255
2016-01-23 12:00:01 BackupPC_nightly now running
BackupPC_refCountUpdate -m -s -c -P 6 -r 0-127
2016-01-23 12:04:55  admin : Missing pool file
0010f3fbc8b305b003800d08a415e9f4 count 1
2016-01-23 12:12:53  admin : Missing pool file
15bf91720300106d0300107fb128e928 count 1
2016-01-23 12:13:22  admin1 : Missing pool file
a20eddff930103001012ef8c93010300 count 1
2016-01-23 12:28:36  admin : BackupPC_refCountPrint: total errors: 2
2016-01-23 12:28:36  admin : xferPids
2016-01-23 12:28:36 BackupPC_nightly now running BackupPC_sendEmail
2016-01-23 12:28:37  admin1 : BackupPC_refCountPrint: total errors: 1
2016-01-23 12:28:37  admin1 : xferPids
2016-01-23 12:28:37 Finished  admin1  (BackupPC_nightly -P 6 128 255)
2016-01-23 12:28:48 Finished  admin  (BackupPC_nightly -m -P 6 0 127)
2016-01-23 12:28:48 Pool nightly clean removed 0 files of size 0.00GB
2016-01-23 12:28:48 Pool is 0.00GB, 0 files (0 repeated, 0 max chain,
0 max links), 0 directories
2016-01-23 12:28:48 Cpool nightly clean removed 0 files of size 0.00GB
2016-01-23 12:28:48 Cpool is 0.00GB, 0 files (0 repeated, 0 max chain,
0 max links), 0 directories
2016-01-23 12:28:48 Pool4 nightly clean removed 0 files of size 0.00GB
2016-01-23 12:28:48 Pool4 is 0.00GB, 0 files (0 repeated, 0 max chain,
0 max links), 0 directories
2016-01-23 12:28:48 Cpool4 nightly clean removed 0 files of size 0.00GB
2016-01-23 12:28:48 Cpool4 is 281.26GB, 5227397 files (0 repeated, 0
max chain, 144350 max links), 16512 directories
2016-01-23 12:28:48 Running BackupPC_rrdUpdate (pid=5069)
2016-01-23 12:28:50  admin-1 : /usr/bin/rrdtool is not a valid executable
2016-01-23 12:28:50 Finished  admin-1  (BackupPC_rrdUpdate)
2016-01-23 12:42:36 Finished full backup on srv1
2016-01-23 23:00:00 Next wakeup is 2016-01-24 00:00:00
2016-01-23 23:43:08 Started incr backup on srv1 (pid=5411, share=full)
2016-01-23 23:44:50 Started incr backup on srv9 (pid=5410, share=full)
2016-01-24 00:00:00 Reading hosts file
2016-01-24 00:00:00 Next wakeup is 2016-01-24 01:00:00
2016-01-24 00:10:22 Re-read config file because of a SIG_HUP
2016-01-24 00:10:22 Next wakeup is 2016-01-24 01:00:00
2016-01-24 01:00:00 Reading hosts file
2016-01-24 01:00:00 Next wakeup is 2016-01-24 02:00:00
2016-01-24 01:21:25 srv1: removing filled backup 0
2016-01-24 01:59:46 srv9: removing filled backup 0
2016-01-24 02:00:00 Next wakeup is 2016-01-24 03:00:00
2016-01-24 03:00:00 Next wakeup is 2016-01-24 04:00:00
2016-01-24 04:00:00 Next wakeup is 2016-01-24 05:00:00
2016-01-24 05:00:00 Next wakeup is 2016-01-24 06:00:00
2016-01-24 06:00:00 Next wakeup is 2016-01-24 07:00:00
2016-01-24 07:00:00 Next wakeup is 2016-01-24 12:00:00
2016-01-24 07:50:12 Finished incr backup on srv9
2016-01-24 08:13:52 Finished incr backup on srv1


as you can see, @2016-01-24 01:21:25 and @2016-01-24 01:59:46 BPC has
removed #0 from both servers.

I don't think this is normal. I'm started from scratch for the third
(or fourth) time, with no pool files or previous dumps. Its a plain
installation of BPC and something is not working properly.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Adam Goryachev]

On 24/01/2016 20:48, Gandalf Corvotempesta wrote:
> 2016-01-23 12:10 GMT+01:00 Gandalf Corvotempesta
> :
>> Let's see what would happen this night.
> The same happened again.
>
> 2016-01-22 22:41:19 Created directory /var/backups/backuppc/pc/srv1/refCnt
> 2016-01-22 22:41:19 full backup started for directory full
> 2016-01-23 11:59:06 full backup 0 complete, 4628296 files, 4628296
> bytes, 1 xferErrs (0 bad files, 0 bad shares, 1 other)
> 2016-01-23 23:43:08 incr backup started for directory full
> 2016-01-24 01:21:25 incr backup 1 complete, 4630086 files, 4630086
> bytes, 1 xferErrs (0 bad files, 0 bad shares, 1 other)
>
> Now #0 is missing. I only have #1 set as incremental.
>

Could you please try changing the share name to something other than full.

Regards,
Adam


--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-24 10:48 GMT+01:00 Gandalf Corvotempesta
:
> I don't think this is normal. I'm started from scratch for the third
> (or fourth) time, with no pool files or previous dumps. Its a plain
> installation of BPC and something is not working properly.

Another full is started. This is not OK, it should start an
incremental, but BPC has removed #0 (it was a full),
thus now is always running full

2016-01-22 22:41:19 Created directory /var/backups/backuppc/pc/srv1/refCnt
2016-01-22 22:41:19 full backup started for directory full
2016-01-23 11:59:06 full backup 0 complete, 4628296 files, 4628296
bytes, 1 xferErrs (0 bad files, 0 bad shares, 1 other)
2016-01-23 23:43:08 incr backup started for directory full
2016-01-24 01:21:25 incr backup 1 complete, 4630086 files, 4630086
bytes, 1 xferErrs (0 bad files, 0 bad shares, 1 other)
2016-01-24 23:00:04 full backup started for directory full


Why BPC is removing all #0 backups? Every time.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-24 23:17 GMT+01:00 Adam Goryachev :
> Could you please try changing the share name to something other than full.

I'll try tomorrow morning. I would like to keep this backup running.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-21 23:46 GMT+01:00 Adam Goryachev :
> Then you have some other error causing problems. Check various things
> such as permissions, disk space, errors on the client, etc. Something is
> causing rsync to fail, and you will need to fix that to get successful
> backups. Check both system logs as well as rsync logs on both server and
> client.

Started from scratch another time.
This time with XFS and not Ext4.

srv2 took 638.1 minutes to created first full. This is good. no rsync
errors in log file:

2016-01-22 23:00:01 Created directory /var/backups/backuppc/pc/srv2/refCnt
2016-01-22 23:00:01 full backup started for directory full
2016-01-23 09:38:07 full backup 0 complete, 2912565 files, 2912565
bytes, 1 xferErrs (0 bad files, 0 bad shares, 1 other)

As expected there is no "needFsck.dump" file, so on next run, fsck
should not run.

Actually, srv2 should be in a clean state, if on next run BPC would
remove backup #0, there is a bug in BPC.

The same is happing for srv1 (backup is still running, "count update #3/5"):

2016-01-22 22:41:19 Created directory /var/backups/backuppc/pc/srv1/refCnt
2016-01-22 22:41:19 full backup started for directory full
2016-01-23 11:59:06 full backup 0 complete, 4628296 files, 4628296
bytes, 1 xferErrs (0 bad files, 0 bad shares, 1 other)

Let's see what would happen this night.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-21 12:41 GMT+01:00 Adam Goryachev :
> Try changing the sharename, see if it makes a difference. You could try
> "complete" or similar. Worst case, it makes no difference, but best
> case, it will solve your problem, and provide a big clue about a bug.

Actually i have some backups running. I'll change where these are finished.

Another strange thing that i've found:

#0 does not exist
#1 is incremental. But transfered files number is 2892544 (almost the
whole server). This is ok if an incremental was made based on an empty
set (#0 does not exists, so all file where transferred)
#3 is another incremental but files number is 2896236. That's too
much. Almost the same as #1

I see these in "File Size/Count Reuse Summary" table on web interface.
Are these number referred to a complete backup and not to the selected backup ?

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-21 11:44 GMT+01:00 Adam Goryachev :
> Silly question but have you named the host full or the share full?
> A lot of your logs that you posted seemed to have full as one of those
> names. It could be confusing bpc and it's unlikely that anyone has
> tested that scenario.


Share is named "full" as point to the whole server.
So, "Backup full" means that is backiung up the share "full" ? That makes sense.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Adam Goryachev]

On 21/01/2016 22:22, Gandalf Corvotempesta wrote:
> 2016-01-21 12:16 GMT+01:00 Adam Goryachev 
> :
>> OK, I'd suggest to try changing this to something else. Usually I would
>> backup / which obviously means the whole server (ie, root folder and
>> everything under it). Though I also tend to specify to only backup the
>> single filesystem to avoid backups of /proc or /sys and so on. So if you
>> have multiple mount points, then you need to specify each one as a "share".
> /proc and /sys are already excluded as well tmp and something else
>
>> Are you using rsyncd as opposed to rsync? That is really the only method
>> I can think of to have the "share name" not equal to a actual directory
>> or path
> I'm using rsyncd.
>
Try changing the sharename, see if it makes a difference. You could try 
"complete" or similar. Worst case, it makes no difference, but best 
case, it will solve your problem, and provide a big clue about a bug.

Regards,
Adam

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Adam Goryachev]

On 21 January 2016 8:42:23 pm AEDT, Gandalf Corvotempesta 
 wrote:
>2016-01-21 10:23 GMT+01:00 Gandalf Corvotempesta
>:
>> This is what I see from control panel during copying #5 => #4 (right
>now)
>
>Copy finished now is running a backup (seems to be an incremental)
>In status column I see: "backup full" and in Type column: "incr"
>
>What does "backup full" means if type is "incr" ?
>
Silly question but have you named the host full or the share full?
A lot of your logs that you posted seemed to have full as one of those 
names. It could be confusing bpc and it's unlikely that anyone has 
tested that scenario.

Regards
Adam

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-21 10:23 GMT+01:00 Gandalf Corvotempesta
:
> This is what I see from control panel during copying #5 => #4 (right now)

Copy finished now is running a backup (seems to be an incremental)
In status column I see: "backup full" and in Type column: "incr"

What does "backup full" means if type is "incr" ?

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Les Mikesell]

On Thu, Jan 21, 2016 at 3:23 AM, Gandalf Corvotempesta
 wrote:
>
> This is what I see from control panel during copying #5 => #4 (right now)
>
> If I understood properly, only last backup should be filled. This
> seems ok in my environment,
> as all except #5 are not filled (filled=0). #5 is filled (filled=1)
>
> But #1 and #3 should not be incremental, there is no full to use as
> starting point for their, or, better,
> there was a full in #0, but #0 is disappeared like #2

V4 does it backwards from v3.   The last backup is always filled and
the older ones are changed to reverse deltas.  It must move/copy
things around to arrange that.   And the full and incremental runs
aren't tied to keeping filled/unfilled backups anymore.  But, I still
don't see why any expired already.

-- 
   Les Mikesell
 lesmikes...@gmail.com

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-21 16:30 GMT+01:00 Les Mikesell :
> V4 does it backwards from v3.   The last backup is always filled and
> the older ones are changed to reverse deltas.  It must move/copy
> things around to arrange that.   And the full and incremental runs
> aren't tied to keeping filled/unfilled backups anymore.  But, I still
> don't see why any expired already.

Another this to know is why BPC is fsck every backups every time.
After each backup, it will run fsck. It should run just for the last
executed backup, as previously ones was alreday checked.

But is running fsck every time for each backup that I have, taking
many many hours. Like now:
srv1 is fsck #1, then #2, then #4. after all, it will run a new backup
and start again the fsck

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Les Mikesell]

On Thu, Jan 21, 2016 at 10:13 AM, Gandalf Corvotempesta
 wrote:
> 2016-01-21 16:30 GMT+01:00 Les Mikesell :
>> V4 does it backwards from v3.   The last backup is always filled and
>> the older ones are changed to reverse deltas.  It must move/copy
>> things around to arrange that.   And the full and incremental runs
>> aren't tied to keeping filled/unfilled backups anymore.  But, I still
>> don't see why any expired already.
>
> Another this to know is why BPC is fsck every backups every time.
> After each backup, it will run fsck. It should run just for the last
> executed backup, as previously ones was alreday checked.
>
> But is running fsck every time for each backup that I have, taking
> many many hours. Like now:
> srv1 is fsck #1, then #2, then #4. after all, it will run a new backup
> and start again the fsck
>

I see some earlier mail list messages saying that is normal:
http://sourceforge.net/p/backuppc/mailman/message/34478542/
I guess that's when it removes files that are not in any current
backup.  Apparently it only takes a long time when you have a very
large number of files.

-- 
   Les Mikesell
  lesmikes...@gmail.com

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-21 18:43 GMT+01:00 Les Mikesell :
> I see some earlier mail list messages saying that is normal:
> http://sourceforge.net/p/backuppc/mailman/message/34478542/
> I guess that's when it removes files that are not in any current
> backup.  Apparently it only takes a long time when you have a very
> large number of files.

I understood what is happening by looking at the code. I need help to
debug more.

When BackupPC_dump is running, a temporary "needFsck.dump" is created
(line #985)
This will trigger an fsck on next run (lines #723-#730)

Temporary file is deleted at #1159 but in my case this is not
happening, thus, an fsck is always run.

So, in my case, the issue is between lines #985 and #1159

Probably, my backups are always detected as failed. A failed backup
will exit without removing the temporary fsck file (line #1152)


Somethimes I'm getting these errors:

2016-01-21 13:53:09 Got fatal error during xfer (rsync error: error in
rsync protocol data stream (code 12) at io.c(629) [generator=3.0.9.3])
2016-01-21 13:53:15 Backup aborted (rsync error: error in rsync
protocol data stream (code 12) at io.c(629) [generator=3.0.9.3])

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-21 23:58 GMT+01:00 Stephen :
> I have glanced at the code and also believe that BackupPC_fsck is running
> unnecessarily after every backup attempt, whether it is successful or not.

Please check if you have a file called "needFsck.dump" in your refCnt directory.
This file is created when backups starts and should be automatically
deleted when it ends.

I think that for some reason this file is never removed and thus an
fsck is always forced.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Adam Goryachev]

On 22/01/16 09:58, Stephen wrote:
> I have glanced at the code and also believe that BackupPC_fsck is running
> unnecessarily after every backup attempt, whether it is successful or not.
>
> In my xferLogs, BackupPC_refCountUpdate is being called twice at the end of
> a backup. Once like this:
>
> Xfer PIDs are now
> Running BackupPC_refCountUpdate -h afsgaia1.cas.unc.edu on somehost.unc.edu
> xferPids 4508
> BackupPC_refCountUpdate: host somehost.unc.edu got 0 errors
> BackupPC_refCountPrint: total errors: 0
> xferPids
> Finished BackupPC_refCountUpdate (running time: 16 sec)
>
> Then again like this:
> Running BackupPC_refCountUpdate -h somehost.unc.edu -f -c on somehost.unc.edu
> xferPids 4509
> BackupPC_refCountUpdate: host somehost.unc.edu got 0 errors
> BackupPC_refCountPrint: total errors: 0
> xferPids
> Finished BackupPC_refCountUpdate (running time: 1334 sec)
>
> The second refCountUpdate includes the "-f -c" args which appear to force
> an fsck on the host.
>
> I'm using rsync over ssh and there are no errors reported:
>
> Done: 0 errors, 31 filesExist, 86366561 sizeExist, 47163323 sizeExistComp,
> 0 filesTotal, 0 sizeTotal, 49 filesNew, 180552022 sizeNew, 58504938
> sizeNewComp, 242121 inode
> Number of files: 105259
> Number of files transferred: 179
> Total file size: 2823131278 bytes
> Total transferred file size: 292936438 bytes
> Literal data: 18830521 bytes
> Matched data: 248088062 bytes
> File list size: 2308933
> File list generation time: 0.001 seconds
> File list transfer time: 0.000 seconds
> Total bytes sent: 616768
> Total bytes received: 21567585
> sent 616768 bytes  received 21567585 bytes  36638.07 bytes/sec
> total size is 2823131278  speedup is 127.26
> DoneGen: 0 errors, 2 filesExist, 8306 sizeExist, 61440 sizeExistComp, 90549
> filesTotal, 2823131278 sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp,
> 242138 inode
>
> Unlike Gandalf, my backups are succeeding. I have successfully performed
> restores. Fulls and incrs happen as expected. Expiration looks correct per
> my config policy.

Yes, exactly. It seems to run the full fsck even when not required. 
Preventing the second run is probably easy, but working out why it was 
written that way, and ensuring it really isn't needed, that is harder.

If you have some extra time to look into the code, then I'm sure many 
people would appreciate any analysis you can do.

Regards,
Adam

-- 
Adam Goryachev Website Managers www.websitemanagers.com.au

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-21 23:09 GMT+01:00 Adam Goryachev :
> Can you run dmesg, and see if you have any lines like this:
> [11613050.504117] rsync_bpc[7279]: segfault at 7f9ee5c7e428 ip
> 004473af sp 7ffc3d7bdf80 error 4 in rsync_bpc[40+75000]
>
> There seems to be some bug in rsync_bpc, I was working on tracking that
> down last week, but my C programming is rather limited, so I'm stuck.
> Hoping someone else on the bpc-dev list might be able to assist.
> Otherwise, I might try one of the online programming forums to see if I
> can get some assistance there.

I don't have any lines like yours.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Stephen]

I have glanced at the code and also believe that BackupPC_fsck is running 
unnecessarily after every backup attempt, whether it is successful or not.

In my xferLogs, BackupPC_refCountUpdate is being called twice at the end of 
a backup. Once like this:

Xfer PIDs are now
Running BackupPC_refCountUpdate -h afsgaia1.cas.unc.edu on somehost.unc.edu
xferPids 4508
BackupPC_refCountUpdate: host somehost.unc.edu got 0 errors
BackupPC_refCountPrint: total errors: 0
xferPids
Finished BackupPC_refCountUpdate (running time: 16 sec)

Then again like this:
Running BackupPC_refCountUpdate -h somehost.unc.edu -f -c on somehost.unc.edu
xferPids 4509
BackupPC_refCountUpdate: host somehost.unc.edu got 0 errors
BackupPC_refCountPrint: total errors: 0
xferPids
Finished BackupPC_refCountUpdate (running time: 1334 sec)

The second refCountUpdate includes the "-f -c" args which appear to force 
an fsck on the host.

I'm using rsync over ssh and there are no errors reported:

Done: 0 errors, 31 filesExist, 86366561 sizeExist, 47163323 sizeExistComp, 
0 filesTotal, 0 sizeTotal, 49 filesNew, 180552022 sizeNew, 58504938 
sizeNewComp, 242121 inode
Number of files: 105259
Number of files transferred: 179
Total file size: 2823131278 bytes
Total transferred file size: 292936438 bytes
Literal data: 18830521 bytes
Matched data: 248088062 bytes
File list size: 2308933
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 616768
Total bytes received: 21567585
sent 616768 bytes  received 21567585 bytes  36638.07 bytes/sec
total size is 2823131278  speedup is 127.26
DoneGen: 0 errors, 2 filesExist, 8306 sizeExist, 61440 sizeExistComp, 90549 
filesTotal, 2823131278 sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp, 
242138 inode

Unlike Gandalf, my backups are succeeding. I have successfully performed 
restores. Fulls and incrs happen as expected. Expiration looks correct per 
my config policy.



On Thu, 21 Jan 2016, Gandalf Corvotempesta wrote:

> 2016-01-21 23:09 GMT+01:00 Adam Goryachev 
> :
>> Can you run dmesg, and see if you have any lines like this:
>> [11613050.504117] rsync_bpc[7279]: segfault at 7f9ee5c7e428 ip
>> 004473af sp 7ffc3d7bdf80 error 4 in rsync_bpc[40+75000]
>>
>> There seems to be some bug in rsync_bpc, I was working on tracking that
>> down last week, but my C programming is rather limited, so I'm stuck.
>> Hoping someone else on the bpc-dev list might be able to assist.
>> Otherwise, I might try one of the online programming forums to see if I
>> can get some assistance there.
>
> I don't have any lines like yours.
>
> --
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
> ___
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
>

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Adam Goryachev]

On 22/01/16 05:45, Gandalf Corvotempesta wrote:
> 2016-01-21 18:43 GMT+01:00 Les Mikesell :
>> I see some earlier mail list messages saying that is normal:
>> http://sourceforge.net/p/backuppc/mailman/message/34478542/
>> I guess that's when it removes files that are not in any current
>> backup.  Apparently it only takes a long time when you have a very
>> large number of files.
> I understood what is happening by looking at the code. I need help to
> debug more.
>
> When BackupPC_dump is running, a temporary "needFsck.dump" is created
> (line #985)
> This will trigger an fsck on next run (lines #723-#730)
>
> Temporary file is deleted at #1159 but in my case this is not
> happening, thus, an fsck is always run.
>
> So, in my case, the issue is between lines #985 and #1159
>
> Probably, my backups are always detected as failed. A failed backup
> will exit without removing the temporary fsck file (line #1152)
>
Which would explain the "missing" backup numbers... possibly. I think if 
you get an error, then you will end up with a missing backup number, so 
you won't have consecutive backup numbers (even if backups haven't been 
expired).

> Somethimes I'm getting these errors:
>
> 2016-01-21 13:53:09 Got fatal error during xfer (rsync error: error in
> rsync protocol data stream (code 12) at io.c(629) [generator=3.0.9.3])
> 2016-01-21 13:53:15 Backup aborted (rsync error: error in rsync
> protocol data stream (code 12) at io.c(629) [generator=3.0.9.3])
Can you run dmesg, and see if you have any lines like this:
[11613050.504117] rsync_bpc[7279]: segfault at 7f9ee5c7e428 ip 
004473af sp 7ffc3d7bdf80 error 4 in rsync_bpc[40+75000]

There seems to be some bug in rsync_bpc, I was working on tracking that 
down last week, but my C programming is rather limited, so I'm stuck. 
Hoping someone else on the bpc-dev list might be able to assist. 
Otherwise, I might try one of the online programming forums to see if I 
can get some assistance there.

Regards,
Adam

-- 
Adam Goryachev Website Managers www.websitemanagers.com.au

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Adam Goryachev]

On 22/01/16 04:43, Les Mikesell wrote:
> On Thu, Jan 21, 2016 at 10:13 AM, Gandalf Corvotempesta
>  wrote:
>> 2016-01-21 16:30 GMT+01:00 Les Mikesell :
>>> V4 does it backwards from v3.   The last backup is always filled and
>>> the older ones are changed to reverse deltas.  It must move/copy
>>> things around to arrange that.   And the full and incremental runs
>>> aren't tied to keeping filled/unfilled backups anymore.  But, I still
>>> don't see why any expired already.
>> Another this to know is why BPC is fsck every backups every time.
>> After each backup, it will run fsck. It should run just for the last
>> executed backup, as previously ones was alreday checked.
>>
>> But is running fsck every time for each backup that I have, taking
>> many many hours. Like now:
>> srv1 is fsck #1, then #2, then #4. after all, it will run a new backup
>> and start again the fsck
>>
> I see some earlier mail list messages saying that is normal:
> http://sourceforge.net/p/backuppc/mailman/message/34478542/
> I guess that's when it removes files that are not in any current
> backup.  Apparently it only takes a long time when you have a very
> large number of files.
>
Yes, it is "normal" but it doesn't seem to be ideal, and I don't think 
it should be required. I started looking at this, but then got side 
tracked with the more serious bug in rsync_bpc. Will eventually come 
back to this, but it means trying to understand the underlying 
principles of how BPC works on the pool and pc trees.

Regards,
Adam

-- 
Adam Goryachev Website Managers www.websitemanagers.com.au

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Adam Goryachev]

On 22/01/16 09:20, Gandalf Corvotempesta wrote:
> 2016-01-21 23:09 GMT+01:00 Adam Goryachev 
> :
>> Can you run dmesg, and see if you have any lines like this:
>> [11613050.504117] rsync_bpc[7279]: segfault at 7f9ee5c7e428 ip
>> 004473af sp 7ffc3d7bdf80 error 4 in rsync_bpc[40+75000]
>>
>> There seems to be some bug in rsync_bpc, I was working on tracking that
>> down last week, but my C programming is rather limited, so I'm stuck.
>> Hoping someone else on the bpc-dev list might be able to assist.
>> Otherwise, I might try one of the online programming forums to see if I
>> can get some assistance there.
> I don't have any lines like yours.
Then you have some other error causing problems. Check various things 
such as permissions, disk space, errors on the client, etc. Something is 
causing rsync to fail, and you will need to fix that to get successful 
backups. Check both system logs as well as rsync logs on both server and 
client.

Regards,
Adam


-- 
Adam Goryachev Website Managers www.websitemanagers.com.au

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-19 11:03 GMT+01:00 Gandalf Corvotempesta
:
> As expected, #2 now is missing. I have #1 (incremental), #3 (incremental)

 Now, another full is running for srv2.

I have:

SRV2:
#1 incremental
#2 incremental
#4 active (is a full)
#0 and #3 are missing.

SRV1:
#1 incremental
#3 incremental
#0, #2, #4 are missing.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-18 16:40 GMT+01:00 Gandalf Corvotempesta
:
> Exactly. I'm trying to debug why #0 disappeared.
> The same happend with both server. The same happened even some days
> ago (where all of my issue started)

Another backup is running.

now I have these statuses in web interface:
srv1: "backup full"
srv2: "merge #2 -> 1"

srv1 log:

2016-01-16 01:32:24 full backup started for directory full
2016-01-16 14:55:54 full backup 0 complete, 4588991 files, 4588991
bytes, 3 xferErrs (0 bad files, 0 bad shares, 3 other)
2016-01-17 05:44:10 incr backup started for directory full
2016-01-17 15:08:04 incr backup 1 complete, 4594227 files, 4594227
bytes, 1 xferErrs (0 bad files, 0 bad shares, 1 other)
2016-01-18 06:00:04 full backup started for directory full
2016-01-19 01:32:18 full backup 2 complete, 4597808 files, 4597808
bytes, 11 xferErrs (0 bad files, 0 bad shares, 11 other)
2016-01-19 09:10:06 incr backup started for directory full

srv2 log:

2016-01-16 01:32:29 full backup started for directory full
2016-01-16 10:51:25 full backup 0 complete, 2895473 files, 2895473
bytes, 13 xferErrs (0 bad files, 0 bad shares, 13 other)
2016-01-17 04:41:21 incr backup started for directory full
2016-01-17 20:20:46 incr backup 1 complete, 2892544 files, 2892544
bytes, 0 xferErrs (0 bad files, 0 bad shares, 0 other)
2016-01-18 04:00:03 full backup started for directory full
2016-01-18 21:42:57 full backup 2 complete, 2888912 files, 2888912
bytes, 1 xferErrs (0 bad files, 0 bad shares, 1 other)
2016-01-19 05:38:12 incr backup started for directory full
2016-01-19 09:09:51 incr backup 3 complete, 2896236 files, 2896236
bytes, 1 xferErrs (0 bad files, 0 bad shares, 1 other)


Backup #0 is still missing from both clients.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-19 9:39 GMT+01:00 Gandalf Corvotempesta
:
> srv2: "merge #2 -> 1"

As expected, #2 now is missing. I have #1 (incremental), #3 (incremental)

there are some issues As I'm using standard BPC4
configuration, I think that there are some bugs.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Les Mikesell]

On Tue, Jan 19, 2016 at 4:03 AM, Gandalf Corvotempesta
 wrote:
> 2016-01-19 9:39 GMT+01:00 Gandalf Corvotempesta
> :
>> srv2: "merge #2 -> 1"
>
> As expected, #2 now is missing. I have #1 (incremental), #3 (incremental)
>
> there are some issues As I'm using standard BPC4
> configuration, I think that there are some bugs.
>

I'd bump up FullKeepCntMin and IncrKeepCntMin to the numbers you want
to see if that keeps them from being expired early.  I always did that
with v3 too just in case something odd happened with the system clock
and to keep backups after decommissioning a host.

Also, since the convention for expiry parameters is
"FullKeepPeriod/FullKeepCnt" etc refer to *Filled* backups, and
"IncrKeepPeriod/IncrKeepCnt" refer to "Unfilled" backups if you change
the scheduling you may need to adjust the FillCycle setting.

-- 
   Les Mikesell
  lesmikes...@gmail.com

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-19 17:01 GMT+01:00 Les Mikesell :
> I'd bump up FullKeepCntMin and IncrKeepCntMin to the numbers you want
> to see if that keeps them from being expired early.  I always did that
> with v3 too just in case something odd happened with the system clock
> and to keep backups after decommissioning a host.

Bumped Full to 3 (it was 1) and Incremental to 21 (it was 7)

But, at least for Full, this is a great waste of space.

> Also, since the convention for expiry parameters is
> "FullKeepPeriod/FullKeepCnt" etc refer to *Filled* backups, and
> "IncrKeepPeriod/IncrKeepCnt" refer to "Unfilled" backups if you change
> the scheduling you may need to adjust the FillCycle setting.

So, what do you suggest ? I would like to have 1 full every 30 days
and 1 incremental each day
being able to revert to each day up to full (30 days before)

I can also try with 1 full every 15 days, having 2 full each month,
but this will double the used space.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Les Mikesell]

On Tue, Jan 19, 2016 at 10:10 AM, Gandalf Corvotempesta
 wrote:
> 2016-01-19 17:01 GMT+01:00 Les Mikesell :
>> I'd bump up FullKeepCntMin and IncrKeepCntMin to the numbers you want
>> to see if that keeps them from being expired early.  I always did that
>> with v3 too just in case something odd happened with the system clock
>> and to keep backups after decommissioning a host.
>
> Bumped Full to 3 (it was 1) and Incremental to 21 (it was 7)
>
> But, at least for Full, this is a great waste of space.
>
>> Also, since the convention for expiry parameters is
>> "FullKeepPeriod/FullKeepCnt" etc refer to *Filled* backups, and
>> "IncrKeepPeriod/IncrKeepCnt" refer to "Unfilled" backups if you change
>> the scheduling you may need to adjust the FillCycle setting.
>
> So, what do you suggest ? I would like to have 1 full every 30 days
> and 1 incremental each day
> being able to revert to each day up to full (30 days before)
>
> I can also try with 1 full every 15 days, having 2 full each month,
> but this will double the used space.

The space usage will only be affected by files that are
changed/deleted between runs.  Any file that still has identical
content will be pooled into one stored instance regardless of how many
backups contain it.   That's the main advantage of BPC over other
systems.

-- 
   Les Mikesell
  lesmikes...@gmail.com

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-19 17:31 GMT+01:00 Les Mikesell :
> The space usage will only be affected by files that are
> changed/deleted between runs.  Any file that still has identical
> content will be pooled into one stored instance regardless of how many
> backups contain it.   That's the main advantage of BPC over other
> systems.

Ok, let's try with 1 full every 15 days and 1 incremental every other
days. Up to 30 backups

Could you suggest a config ? There is something wrong with mine.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-18 2:46 GMT+01:00 Adam Goryachev :
> It would be interesting to hear an update of where you are at, and what
> has happened (either good or bad).

Ok, today is the third day of backup running "properly".

Some things are not clear:

2016-01-16 01:32:24 full backup started for directory full
2016-01-16 14:55:54 full backup 0 complete, 4588991 files, 4588991
bytes, 3 xferErrs (0 bad files, 0 bad shares, 3 other)

2016-01-17 05:44:10 incr backup started for directory full
2016-01-17 15:08:04 incr backup 1 complete, 4594227 files, 4594227
bytes, 1 xferErrs (0 bad files, 0 bad shares, 1 other)

2016-01-18 06:00:04 full backup started for directory full

As you can see, I did a full backup 2016-01-16 01:32:24, then an
incremental on 2016-01-17 05:44:10. The third full @ 2016-01-18
06:00:04 is not normal.

I have this in my configuration:

$Conf{FullPeriod} = 27.97;
$Conf{FullKeepCnt} = 1;
$Conf{FullKeepCntMin} = 1;
$Conf{FullAgeMax} = 60;

$Conf{IncrPeriod} = 0.97;
$Conf{IncrKeepCnt} = 31;
$Conf{IncrKeepCntMin} = 7;
$Conf{IncrAgeMax} = 35;

$Conf{FillCycle} = 0;

so, it should create 1 full every 30 days (more or less), keeping just
1 full. All other days, it should create an incremental.

Why is creating a new full after 2 days the first one ?

Additionally, yesterday I had backup #0 set as full and #1 running as
incremental.
Now, #0 is not more available (is missing from web interface), #1 is
an incremental and #2 is active (is doing a full right now)

Exactly the same happens on a second client.

What I saw is that yesterday, it was copying #1 > #0. Is this normal ?

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Les Mikesell]

On Mon, Jan 18, 2016 at 12:25 PM, Alexander Moisseev
 wrote:
> On 18.01.16 19:53, Les Mikesell wrote:
>> Does anyone understand the docs at
>> http://backuppc.sourceforge.net/BackupPC-4.0.0alpha3_doc.html
>> for $Conf{FillCycle} ?   It looks like expiring is really based on
>> 'filled' backups, not necessarily full backups as in v3.  The part I
>> don't get is that it says the most recent backup is always filled
>> whether it is full or incremental.  If the most recent is filled, how
>> would any ever be unfilled?
>>
>
> In v4 full/incremental related to backup transfer only, filled/unfilled 
> related to storage.
>
> Here is a short explanation:
> http://backuppc.sourceforge.net/BackupPC-4.0.0alpha3_doc.html#BackupPC-4.0
>
> "Backups are stored as "reverse deltas" - the most recent backup is always 
> filled and older backups are reconstitued by merging all the deltas starting 
> with the nearest future filled backup and wworking backwards.
>
> This is the opposite of V3 where incrementals are stored as "forward deltas" 
> to a prior backup (typically the last full backup or prior lower-level 
> incremental backup, or the last full in the case of rsync)."
>
>
> So, the latest backup should always be filled, i.e. it actually contains all 
> files.  After the last backup will be taken, files that latest backup 
> contains will be removed from backups between the last backup and previous 
> filled backup recursively. Until the last backup was taken, previous backup 
> was filled, but now it is just "reverse delta".

OK, but then how does that relate to the expiring part - which it says
is really based on the filled status?

--
   Les Mikesell
  lesmikes...@gmail.com

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-18 17:07 GMT+01:00 Les Mikesell :
> Why don't you put back the default settings to see they work as
> expected?

Because i'm using default settings except posted lines.

I wont decrease full interval, not for waste of space but to not
transfer everything from the client, that is resource heavy.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Alexander Moisseev]

On 18.01.16 21:33, Les Mikesell wrote:
> On Mon, Jan 18, 2016 at 12:25 PM, Alexander Moisseev
>  wrote:
>> On 18.01.16 19:53, Les Mikesell wrote:
>>> Does anyone understand the docs at
>>> http://backuppc.sourceforge.net/BackupPC-4.0.0alpha3_doc.html
>>> for $Conf{FillCycle} ?   It looks like expiring is really based on
>>> 'filled' backups, not necessarily full backups as in v3.  The part I
>>> don't get is that it says the most recent backup is always filled
>>> whether it is full or incremental.  If the most recent is filled, how
>>> would any ever be unfilled?
>>>
>>
>> In v4 full/incremental related to backup transfer only, filled/unfilled 
>> related to storage.
>>
>> Here is a short explanation:
>> http://backuppc.sourceforge.net/BackupPC-4.0.0alpha3_doc.html#BackupPC-4.0
>>
>> "Backups are stored as "reverse deltas" - the most recent backup is always 
>> filled and older backups are reconstitued by merging all the deltas starting 
>> with the nearest future filled backup and wworking backwards.
>>
>> This is the opposite of V3 where incrementals are stored as "forward deltas" 
>> to a prior backup (typically the last full backup or prior lower-level 
>> incremental backup, or the last full in the case of rsync)."
>>
>>
>> So, the latest backup should always be filled, i.e. it actually contains all 
>> files.  After the last backup will be taken, files that latest backup 
>> contains will be removed from backups between the last backup and previous 
>> filled backup recursively. Until the last backup was taken, previous backup 
>> was filled, but now it is just "reverse delta".
>
> OK, but then how does that relate to the expiring part - which it says
> is really based on the filled status?
>

v3: 'full'   shouldn't be expired while there is at least one 'incremental' 
that depends on it
v4: 'filled' shouldn't be expired while there is at least one 'unfilled'
that depends on it

Full/incremental doesn't matter for v4 expiration algorithm since it doesn't 
affect ability to restore particular backup.


--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Alexander Moisseev]

On 16.01.16 22:16, Gandalf Corvotempesta wrote:
> 2016-01-16 19:53 GMT+01:00 Alexander Moisseev :
>> You mentioned earlier you are trying to backup dovecot with Maildir. Do you 
>> have considered to move from Maildir to mdbox? It will definitely help with 
>> BackupPC and rsync performance. More over you will be able to use dovecot 
>> tools (+ your own scripts) or dovecot replication feature instead of 
>> rsync/BackupPC. It will increase performance dramatically.
>
> mdbox is available since version 2. Some server are very old, with dovecot 1.
>
I don't think old hardware will be a problem for version 2.

I am using low end hardware for a small mail server (P4 2.80GHz, 2GB RAM, 
gmirror of 2 SATA drives) and BackupPC server (P4 2.80GHz, 1.5GB RAM, SATA 
drive) and 100 Mbps networking.

About 2 years ago I was using Maildir and BPC v3 (rsyncd with checksum caching).
Mail storage had about 100 thousands files.
The first two full backups was completed in about 4 hours each.
Subsequent full backups was taken about 30 minutes (if there is no concurrent 
backups).
I never tried BPC v4 to backup Maildir.

Then I converted mail storage to mdbox + SIS (single instance attachment 
storage) + compression. I never tried BPC with mdbox, I see no point, but I 
believe file list building should be much quicker since file number is 
significantly lower. I installed dovecot (mdbox + SIS + compression) on a test 
machine with similar hardware configuration. Initial backup with "doveadm 
backup" took about 1-3 hours, I don't remember exact time. Subsequent everyday 
backups take about 1-2 minutes and don't depend on mail storage size or 
messages number, but on changes made between backups.

Finally for the backup end I've installed dovecot (mdbox + SIS + compression) 
on a server with ZFS. Now, in order to have retrospective, I'm taking snapshot 
after each backup and using BackupPC-like aging.
And "zfs send" to make offline copies.

Hostname| Level | Size (MB) | Written (MB) | Used (MB) | Last available   | 
Days old

tank1/vmail | 0 |  27860.45 | 23650.66 |   2128.86 | 2015-07-11 07:51 | 
 191
tank1/vmail | 0 |  27804.61 |  2073.86 |612.73 | 2015-08-12 07:51 | 
 159
tank1/vmail | 0 |  29287.60 |  2227.53 |686.91 | 2015-09-13 07:51 | 
 127
tank1/vmail | 0 |  30896.84 |  2488.86 |601.22 | 2015-10-15 07:51 | 
  95
tank1/vmail | 0 |  35778.12 |  6177.88 |   1852.94 | 2015-11-16 07:51 | 
  63
tank1/vmail | 0 |  37257.94 |  4151.30 |443.62 | 2015-12-18 07:51 | 
  31
tank1/vmail | 0 |  37257.47 |   519.09 |186.52 | 2015-12-22 07:51 | 
  27
tank1/vmail | 0 |  37273.77 |   545.84 |191.96 | 2015-12-26 07:51 | 
  23
tank1/vmail | 0 |  37564.53 |   686.51 |165.02 | 2015-12-30 07:51 | 
  19
tank1/vmail | 0 |  37533.15 |   415.20 |129.96 | 2016-01-03 07:51 | 
  15
tank1/vmail | 0 |  37454.53 |   220.35 | 99.68 | 2016-01-07 07:51 | 
  11
tank1/vmail | 0 |  37419.07 |   170.54 |107.71 | 2016-01-11 07:51 | 
   7
tank1/vmail | 0 |  37520.83 |   317.79 |112.94 | 2016-01-12 07:51 | 
   6
tank1/vmail | 0 |  37633.12 |   328.19 |113.92 | 2016-01-13 07:51 | 
   5
tank1/vmail | 0 |  37792.96 |   345.29 | 83.54 | 2016-01-14 07:51 | 
   4
tank1/vmail | 0 |  37961.14 |   302.97 | 61.54 | 2016-01-15 07:51 | 
   3
tank1/vmail | 0 |  38045.83 |   178.00 | 12.08 | 2016-01-16 07:51 | 
   2
tank1/vmail | 0 |  38060.19 |31.03 | 15.53 | 2016-01-17 07:51 | 
   1
tank1/vmail | 0 |  38029.69 |38.44 |  0.00 | 2016-01-18 07:51 | 
   0
tank1/vmail | - |  38029.69 | 0.00 |  32687.01 |- | 
   -

NAME PROPERTY VALUE
tank1/vmail  used 43.8G
tank1/vmail  logicalused  51.1G
   
Actual total size of mail stored on the mailserver (prior SIS and compression) 
62GB, 170 thousands of messages.
I know it is a very small server, but you can extrapolate the results.


--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Alexander Moisseev]

On 18.01.16 4:46, Adam Goryachev wrote:
> So, while generating the file list might take a long time with BPC v3, I
> don't think it should significantly change the overall backup time.
Assuming checksum caching configured, for first and second backup, yes. But it 
is not true for subsequent backups since most of Maildir files never change 
over time.



--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Adam Goryachev]

On 17/01/16 05:53, Alexander Moisseev wrote:
> On 16.01.16 15:04, Gandalf Corvotempesta wrote:
>> I can't use v3 because i'm having servers with millions of file to be
>> transfered and v3 takes ages in "Building files list".
> You mentioned earlier you are trying to backup dovecot with Maildir. Do you 
> have considered to move from Maildir to mdbox? It will definitely help with 
> BackupPC and rsync performance. More over you will be able to use dovecot 
> tools (+ your own scripts) or dovecot replication feature instead of 
> rsync/BackupPC. It will increase performance dramatically.

I'm not sure of the exact numbers you are referring to (millions is 1 
million or 50 million?, also "ages" is 5 minutes or 5 hours). In any 
case, back in 2012, I did a full backup of one of my mail servers, and 
it was completed in 70 minutes, with approx 1.3 million files (that 
would have been with BPC v3). Equally, a more recent backup with v4 
shows it took 402 minutes (almost 7 hours) and the quickest (recent BPC 
v4) full backup took 146 minutes.

So, while generating the file list might take a long time with BPC v3, I 
don't think it should significantly change the overall backup time. I 
would expect my BPC server has gotten quicker over the years, as has the 
mail server. Variations are likely due to concurrent backups, and 
bandwidth limitations.

It would be interesting to hear an update of where you are at, and what 
has happened (either good or bad).

Regards,
Adam

-- 
Adam Goryachev Website Managers www.websitemanagers.com.au

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Adam Goryachev]

On 16/01/16 23:04, Gandalf Corvotempesta wrote:
> 2016-01-14 23:00 GMT+01:00 Adam Goryachev 
> :
>> I would suggest anyone considering BPC should use v3 unless they
>> specifically need the v4 features, and even then, are prepared for the
>> possible shortcomings inherent in using alpha software.
> I can't use v3 because i'm having servers with millions of file to be
> transfered and v3 takes ages in "Building files list".
> v4 uses native rsync and incremental file list. This is faster,
> transfer start immediatly, not after many many hours.
>
> BTW, 1 server was backuped properly, now is running "fsck #0". As I
> understood from docs, BackupPC should not run when
> fsck in running. I'm having a full dump of srv1, and srv2 that is
> doing "fsck #0". Is this bad or fsck can be run parallel with other
> backups?

AFAIK, when you see fsck #0 on the web interface, it is actually doing a 
refCountUpdate, which just updates the current host, therefore it 
doesn't matter if other hosts are running a concurrent backup.

> Usually how long does it take to complete? this is slowing down the
> other backup that seems to be frozen.

This is dependant on the number of files in the backup, and especially, 
your disk I/O performance. You will probably want to think about RAID10 
or similar for your backup server in order to massively improve 
performance (or use bigger caches, either hardware raid cache, or allow 
linux to cache more). Equally, your filesystem might need tuning, or 
changing to a different FS might help too.

Running multiple concurrent backups might be slower than running one at 
a time, depending on the speed of the client, and the performance of 
your BPC server. Remember, the disk can only read or write from one 
sector at a time, and random performance is significantly worse.

Regards,
Adam

-- 
Adam Goryachev Website Managers www.websitemanagers.com.au

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-14 23:00 GMT+01:00 Adam Goryachev :
> I would suggest anyone considering BPC should use v3 unless they
> specifically need the v4 features, and even then, are prepared for the
> possible shortcomings inherent in using alpha software.

I can't use v3 because i'm having servers with millions of file to be
transfered and v3 takes ages in "Building files list".
v4 uses native rsync and incremental file list. This is faster,
transfer start immediatly, not after many many hours.

BTW, 1 server was backuped properly, now is running "fsck #0". As I
understood from docs, BackupPC should not run when
fsck in running. I'm having a full dump of srv1, and srv2 that is
doing "fsck #0". Is this bad or fsck can be run parallel with other
backups?

Usually how long does it take to complete? this is slowing down the
other backup that seems to be frozen.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-16 19:53 GMT+01:00 Alexander Moisseev :
> You mentioned earlier you are trying to backup dovecot with Maildir. Do you 
> have considered to move from Maildir to mdbox? It will definitely help with 
> BackupPC and rsync performance. More over you will be able to use dovecot 
> tools (+ your own scripts) or dovecot replication feature instead of 
> rsync/BackupPC. It will increase performance dramatically.

mdbox is available since version 2. Some server are very old, with dovecot 1.

Anyway, actually I have 2 full backup completed in 13 and 9 hours,
both started in paralles today at 01:20. Let's see tonight. Two
incremental should start...

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Alexander Moisseev]

On 16.01.16 15:04, Gandalf Corvotempesta wrote:
> I can't use v3 because i'm having servers with millions of file to be
> transfered and v3 takes ages in "Building files list".

You mentioned earlier you are trying to backup dovecot with Maildir. Do you 
have considered to move from Maildir to mdbox? It will definitely help with 
BackupPC and rsync performance. More over you will be able to use dovecot tools 
(+ your own scripts) or dovecot replication feature instead of rsync/BackupPC. 
It will increase performance dramatically.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Holger Parplies]

Hi,

much has been said in this thread, but I believe this has not:
You do realize that BackupPC 4.x is alpha software, right? You're lucky if it
works at all, you should be surprised if it performs well, and you're not
using it in a production environment anyway, right? Ok, the reality might not
be quite as bad, but you need to be aware that version 4 is not a newer
version 3 with some improvements, it is basically a re-write with a whole new
concept behind it.

If you are looking for something stable, I'd recommend BackupPC 3.x, which
you'll undoubtedly also get more and better support for here, if you even
need it. And yes, it *will* be slower than native rsync.

For me (and, I believe, for many others), version 3 (and even version 2!) works
so well that there is no reason to move to version 4.

Hope that helps.

Regards,
Holger

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Les Mikesell]

On Thu, Jan 14, 2016 at 11:28 AM, Holger Parplies  wrote:
>
> For me (and, I believe, for many others), version 3 (and even version 2!) 
> works
> so well that there is no reason to move to version 4.
>

I have to agree with that.  Really about the only issue people have
with v3 is that if you ever want to copy the entire archive tree to a
new system or to make an offsite copy it is a problem for
file-oriented copy methods to recreate the large number of hardlinks.

The changes in v4 may help some situations, but I wonder if the
attempt to locate existing matching content from different locations
might waste time where there are large numbers of tiny files.

If you do try v3, note that using the checksum-seed=32761 option will
speed up full runs but you won't see the difference until the 3rd
full.  That is, the block checksums are cached on the 2nd full copy of
a file and then subsequently used so the server side does not have to
uncompress and recompute them.

-- 
   Les Mikesell
 lesmikes...@gmail.com

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Adam Goryachev]

On 15/01/16 05:55, Les Mikesell wrote:
> On Thu, Jan 14, 2016 at 11:28 AM, Holger Parplies  wrote:
>> For me (and, I believe, for many others), version 3 (and even version 2!) 
>> works
>> so well that there is no reason to move to version 4.
>>
> I have to agree with that.  Really about the only issue people have
> with v3 is that if you ever want to copy the entire archive tree to a
> new system or to make an offsite copy it is a problem for
> file-oriented copy methods to recreate the large number of hardlinks.
>
> The changes in v4 may help some situations, but I wonder if the
> attempt to locate existing matching content from different locations
> might waste time where there are large numbers of tiny files.

I must say that this is the only reason for me choosing to run v4. For 
some reason, a number of systems I need to support backups for have a 
workflow that involves photos taken, downloaded from the camera to the 
server, and then renaming/copying/moving them to various different 
folders. With v4, BPC only downloads them once, and each time they are 
moved/copied to a different folder there is no transfer. This helps my 
backups actually work/complete in a much shorter time.

However, for most other scenarios, I doubt it would make a lot of 
difference.

I would suggest anyone considering BPC should use v3 unless they 
specifically need the v4 features, and even then, are prepared for the 
possible shortcomings inherent in using alpha software.

Regards,
Adam

-- 
Adam Goryachev
Website Managers
P: +61 2 8304 a...@websitemanagers.com.au
F: +61 2 8304 0001 www.websitemanagers.com.au


-- 
Adam Goryachev Website Managers www.websitemanagers.com.au

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-13 21:07 GMT+01:00 Gandalf Corvotempesta
:
> 2016/01/13 20:48:21 [19453] 2016/01/13 20:48:21: host unknown
> (172.17.0.1) send
> home/user/domains/domain.it/public_html/cgi-bin/.htaccess (17 bytes).
> Total 43 bytes.
> 2016/01/13 20:48:21 [19453] 2016/01/13 20:48:21: host unknown
> (172.17.0.1) send
> home/user/domains/domain.it/.htpasswd/.protected.list (0 bytes). Total
> 39 bytes.
> 2016/01/13 21:03:29 [19453] 2016/01/13 21:03:29: host unknown
> (172.17.0.1) send home/user/domains/domain.it/awstats/.htaccess (233
> bytes). Total 276 bytes.
> 2016/01/13 21:03:29 [19453] 2016/01/13 21:03:29: host unknown
> (172.17.0.1) send
> home/user/domains/domain.it/awstats/awstats.domain.it.1003.alldomains.html
> (5934 bytes). Total 5977 bytes.
>
> Totally frozen between 2016/01/13 20:48:21 and 2016/01/13 21:03:29

Raw performance by direct rsync between these two servers:

receiving incremental file list
test.img
  1,073,741,824 100%   25.04MB/s0:00:40 (xfr#1, to-chk=0/1)

sent 77 bytes  received 1,073,873,061 bytes  24,686,738.80 bytes/sec
total size is 1,073,741,824  speedup is 1.00


24.5MB/s, not too much, but not too bad. 24 times faster than BPC
(with BPC i got about 1MB/s)



raw performance writing to disk on BPC server (same partitions used by
BPC as storage):

# dd if=/dev/zero of=/var/backups/test.img bs=1M count=1
^C8815+0 records in
8815+0 records out
9243197440 bytes (9.2 GB) copied, 142.267 s, 65.0 MB/s



So, network is not an issue. "plain" rsync is not an issue, write
performance to disks are not an issue..
The only remaining is BPC.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-12 8:29 GMT+01:00 Patrick Begou :
> I'm using BackupPC 3.3.1 for a while now and backing up a partition of more 
> than
> 100GB  (from a macbook client) was requiering more than 24 hours on a backupPC
> server with only 2GB of RAM. Increasing the RAM to 6GB has totally solved the
> problem. Now a full backup on 180GB requires a little bit more than 3 hours.

Ok, let's start again from the beginning:

# cat /etc/debian_version
8.2

# grep MemTotal /proc/meminfo
MemTotal:8190512 kB

# grep 'model name' /proc/cpuinfo  | head -n1
model name : Intel(R) Xeon(R) CPU   E5410  @ 2.33GHz

# mount | grep backups
/dev/mapper/vg0-lv_backups on /var/backups type ext3
(rw,nosuid,nodev,noexec,noatime,nodiratime,nobarrier)

Click on "start full backup" in admin page

# cat /var/log/BackupPC/LOG
2016-01-13 20:21:26 Running BackupPC_Admin_SCGI (pid=29415)
2016-01-13 20:22:11 Reading hosts file
2016-01-13 20:22:11 BackupPC started, pid 29463
2016-01-13 20:22:11 Running BackupPC_Admin_SCGI (pid=29464)
2016-01-13 20:22:11 Next wakeup is 2016-01-13 23:00:00
2016-01-13 20:28:58 User backuppc requested backup of srv1 (srv1)
2016-01-13 20:28:58 Started full backup on srv1 (pid=29473, share=full)


Let's wait

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-13 20:39 GMT+01:00 Gandalf Corvotempesta
:
> # cat /var/log/BackupPC/LOG
> 2016-01-13 20:21:26 Running BackupPC_Admin_SCGI (pid=29415)
> 2016-01-13 20:22:11 Reading hosts file
> 2016-01-13 20:22:11 BackupPC started, pid 29463
> 2016-01-13 20:22:11 Running BackupPC_Admin_SCGI (pid=29464)
> 2016-01-13 20:22:11 Next wakeup is 2016-01-13 23:00:00
> 2016-01-13 20:28:58 User backuppc requested backup of srv1 (srv1)
> 2016-01-13 20:28:58 Started full backup on srv1 (pid=29473, share=full)

Same issues. This is "rsync.log" client side:

2016/01/13 20:48:21 [19453] 2016/01/13 20:48:21: host unknown
(172.17.0.1) send
home/user/domains/domain.it/public_html/cgi-bin/.htaccess (17 bytes).
Total 43 bytes.
2016/01/13 20:48:21 [19453] 2016/01/13 20:48:21: host unknown
(172.17.0.1) send
home/user/domains/domain.it/.htpasswd/.protected.list (0 bytes). Total
39 bytes.
2016/01/13 21:03:29 [19453] 2016/01/13 21:03:29: host unknown
(172.17.0.1) send home/user/domains/domain.it/awstats/.htaccess (233
bytes). Total 276 bytes.
2016/01/13 21:03:29 [19453] 2016/01/13 21:03:29: host unknown
(172.17.0.1) send
home/user/domains/domain.it/awstats/awstats.domain.it.1003.alldomains.html
(5934 bytes). Total 5977 bytes.

Totally frozen between 2016/01/13 20:48:21 and 2016/01/13 21:03:29

BPC server side was writing to pool (I've seen many activity by using
"strace" to rsync_bpc)

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Les Mikesell]

On Wed, Jan 13, 2016 at 2:52 PM, Gandalf Corvotempesta
 wrote:
> >>
>> Totally frozen between 2016/01/13 20:48:21 and 2016/01/13 21:03:29

Did your strace test show a hanging system call on any of the active
processes in this time?

> raw performance writing to disk on BPC server (same partitions used by
> BPC as storage):
>
> # dd if=/dev/zero of=/var/backups/test.img bs=1M count=1
> ^C8815+0 records in
> 8815+0 records out
> 9243197440 bytes (9.2 GB) copied, 142.267 s, 65.0 MB/s

That's not at all like what rsync would be doing when it merges
changes to a compressed file.

-- 
   Les Mikesell
  lesmikes...@gmail.com

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-13 22:07 GMT+01:00 Les Mikesell :
> Did your strace test show a hanging system call on any of the active
> processes in this time?

Nothing is hanged. When this occurs, no transfer is happening via network
and both "rsync_bpc" processes are parsing tons of these:


FIRST PROCES:
read(5, "82f3016ca8f4b309aa141fb1aee9dfb0"..., 8184) = 4117
select(6, [5], [], NULL, {60, 0})   = 1 (in [5], left {59, 855296})
read(5, "34166a01128e65c0e98ce2a634ea"..., 8184) = 4093
select(6, [5], [], NULL, {60, 0})   = 1 (in [5], left {59, 988318})

SECOND PROCESS:
select(4, [3], [], NULL, {60, 0})   = 1 (in [3], left {59, 34})
read(3, "63db4c25f333ca\0\202\3Vg9\211\247\346\"N\233<\215\320x\4\272"...,
4092) = 2896
select(4, [3], [], NULL, {60, 0})   = 1 (in [3], left {59, 999675})
read(3, 
"197addc7b4\0001\2V\v\v\222\2655\242\213\242\310P\222\343\331\211Q\244T\237"...,
1196) = 1196
select(7, NULL, [6], [6], {60, 0})  = 1 (out [6], left {59, 99})
write(6, "690f3f5dc38571ac1d63db4c25f333ca"..., 4092) = 4092
select(7, NULL, [6], [6], {60, 0})  = 1 (out [6], left {59, 98})
write(6, "\363a\306v}\376\364@\241\236{:A ", 14) = 14
select(4, [3], [], NULL, {60, 0})   = 1 (in [3], left {59, 906237})
read(3, "\374\17\0\7", 4)   = 4
select(4, [3], [], NULL, {60, 0})   = 1 (in [3], left {59, 999806})
read(3, "ff5cd88ec882380bd1ade93a3c24\0M\2V"..., 4092) = 2896


What's the meaning of two rsync_bpc processes?


> That's not at all like what rsync would be doing when it merges
> changes to a compressed file.

I know, but having a slow disk would slow down also rsync and bpc.
This test told me that disks are working properly and bottleneck shold
be somewhere else.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Adam Goryachev]

On 14/01/16 08:14, Gandalf Corvotempesta wrote:
> 2016-01-13 22:07 GMT+01:00 Les Mikesell :
>> Did your strace test show a hanging system call on any of the active
>> processes in this time?
> Nothing is hanged. When this occurs, no transfer is happening via network
> and both "rsync_bpc" processes are parsing tons of these:
>
>
> FIRST PROCES:
> read(5, "82f3016ca8f4b309aa141fb1aee9dfb0"..., 8184) = 4117
> select(6, [5], [], NULL, {60, 0})   = 1 (in [5], left {59, 855296})
> read(5, "34166a01128e65c0e98ce2a634ea"..., 8184) = 4093
> select(6, [5], [], NULL, {60, 0})   = 1 (in [5], left {59, 988318})
>
> SECOND PROCESS:
> select(4, [3], [], NULL, {60, 0})   = 1 (in [3], left {59, 34})
> read(3, "63db4c25f333ca\0\202\3Vg9\211\247\346\"N\233<\215\320x\4\272"...,
> 4092) = 2896
> select(4, [3], [], NULL, {60, 0})   = 1 (in [3], left {59, 999675})
> read(3, 
> "197addc7b4\0001\2V\v\v\222\2655\242\213\242\310P\222\343\331\211Q\244T\237"...,
> 1196) = 1196
> select(7, NULL, [6], [6], {60, 0})  = 1 (out [6], left {59, 99})
> write(6, "690f3f5dc38571ac1d63db4c25f333ca"..., 4092) = 4092
> select(7, NULL, [6], [6], {60, 0})  = 1 (out [6], left {59, 98})
> write(6, "\363a\306v}\376\364@\241\236{:A ", 14) = 14
> select(4, [3], [], NULL, {60, 0})   = 1 (in [3], left {59, 906237})
> read(3, "\374\17\0\7", 4)   = 4
> select(4, [3], [], NULL, {60, 0})   = 1 (in [3], left {59, 999806})
> read(3, "ff5cd88ec882380bd1ade93a3c24\0M\2V"..., 4092) = 2896
>
>
> What's the meaning of two rsync_bpc processes?
>
>
>> That's not at all like what rsync would be doing when it merges
>> changes to a compressed file.
> I know, but having a slow disk would slow down also rsync and bpc.
> This test told me that disks are working properly and bottleneck shold
> be somewhere else.
>

Ummm, really? I think you are confused. Depending on where exactly the 
above processes are reading or writing to (most likely it isn't network, 
which means it is almost certainly your backuppc server disks) will tell 
you where the bottleneck is. You have identified that the client is 
providing the data to the server quickly enough, but the server is too 
slow to process this data (ie, do whatever needs to happen to save it in 
the correct place). This is almost certainly one or more of the 
following reasons:
1) Slow I/O
2) Not enough RAM leading to not enough cache leading to slow I/O
3) Slow CPU

You provided some other information in another email:
> Raw performance by direct rsync between these two servers:
>
> receiving incremental file list
> test.img
>1,073,741,824 100%   25.04MB/s0:00:40 (xfr#1, to-chk=0/1)
>
> sent 77 bytes  received 1,073,873,061 bytes  24,686,738.80 bytes/sec
> total size is 1,073,741,824  speedup is 1.00
>
>
> 24.5MB/s, not too much, but not too bad. 24 times faster than BPC
> (with BPC i got about 1MB/s)
This is completely rubbish, it isn't a useful comparison of anything. I 
am almost certain that your actual client isn't made up of files with a 
average size of 1GB. In fact the snippet of the rsyncd log that you 
previously provided showed very small files. It still isn't a meaningful 
comparison, but at least it is more realistic if you used the actual 
files you are trying to backup, even if it is only a subset of them.

BTW, the reason it isn't so relevant is because backuppc does a lot more 
work on the server side than plain rsync, the client side performance is 
relevant, and could at least show that the client is capable.
> raw performance writing to disk on BPC server (same partitions used by
> BPC as storage):
>
> # dd if=/dev/zero of=/var/backups/test.img bs=1M count=1
> ^C8815+0 records in
> 8815+0 records out
> 9243197440 bytes (9.2 GB) copied, 142.267 s, 65.0 MB/s
Totally irrelevant. BackupPC is doing lots of small random reads and 
writes. However, maybe that is relevant, because 65MB/s on any single 
HDD from the past 5 years, let alone a RAID array is abysmal for 
streaming writes. Even a single drive should be capable of at least 100MB/s.

Here is the same statistic from one of my BPC v3 servers:
dr:/mnt/imagestore# dd if=/dev/zero of=/var/backups/test.img bs=1M 
count=1
1+0 records in
1+0 records out
1048576 bytes (10 GB) copied, 16.135 s, 650 MB/s

This is a LV sitting on a RAID5 array:
md0 : active raid5 sde1[4] sdc1[3] sdd1[2] sdb1[0]
   11720658432 blocks super 1.2 level 5, 512k chunk, algorithm 2 
[4/4] []

Which is using these drives:
Model Family: Western Digital Red (AF)
Device Model: WDC WD40EFRX-68WT0N0
User Capacity:4,000,787,030,016 bytes [4.00 TB]
Sector Sizes: 512 bytes logical, 4096 bytes physical
Rotation Rate:5400 rpm

I'm sure you said you had 7200rpm disks, so you should get even better 
performance for both random r/w as well as streaming writes. Which 
brings me back to my earlier concern that you are using a VM for 
backuppc, it is sharing it's 

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-13 22:14 GMT+01:00 Gandalf Corvotempesta
:
> Nothing is hanged. When this occurs, no transfer is happening via network
> and both "rsync_bpc" processes are parsing tons of these:

Seems to be a write delay. rsync doesn't send new files until BPC has
finished writes to disks.

BTW, by changing compressione level from 3 to 1, seems to be a little faster.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-14 0:06 GMT+01:00 Adam Goryachev :
> However, note that this is happening while BackupPC_nightly is running,
> so there is plenty of other IO happening in the background. (in v3 there
> is a nightly process which checks every file in the pool to see if it is
> still needed or not).

There is no nightly at this time here. I'm running nightly in the
middle of the day :)

$Conf{WakeupSchedule} = [12, 23, 24, 1, 2, 3, 4, 5, 6, 7];

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-14 0:06 GMT+01:00 Adam Goryachev :
> Whoops, that was writing to the SSD for the root FS I was rather
> surprised at the high number, but didn't stop and think properly.
> Please see the revised stat:
> dr:/mnt/imagestore# dd if=/dev/zero of=test.img bs=1M count=1
> 1+0 records in
> 1+0 records out
> 1048576 bytes (10 GB) copied, 81.5089 s, 129 MB/s

This is mine:

# dd if=/dev/zero of=/var/backups/test.img bs=1M count=2000
2000+0 records in
2000+0 records out
2097152000 bytes (2.1 GB) copied, 15.4692 s, 136 MB/s


>> I'm sure you said you had 7200rpm disks, so you should get even better
>> performance for both random r/w as well as streaming writes. Which
>> brings me back to my earlier concern that you are using a VM for
>> backuppc, it is sharing it's performance with other things, which
>> works very poorly when dealing with spinning disks (even a streaming
>> write like your example is mixed with other random I/O which means it
>> kills performance).

I'm not using a VM (please don't make confusion between users), It's a
DELL PowerEdge 2950 with 8GB ram, hardware raid and 6x2TB @7200 disks
in RAID-5

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Adam Goryachev]

On 14/01/16 09:29, Adam Goryachev wrote:
> On 14/01/16 08:14, Gandalf Corvotempesta wrote:
>
>> raw performance writing to disk on BPC server (same partitions used by
>> BPC as storage):
>>
>> # dd if=/dev/zero of=/var/backups/test.img bs=1M count=1
>> ^C8815+0 records in
>> 8815+0 records out
>> 9243197440 bytes (9.2 GB) copied, 142.267 s, 65.0 MB/s
> Totally irrelevant. BackupPC is doing lots of small random reads and 
> writes. However, maybe that is relevant, because 65MB/s on any single 
> HDD from the past 5 years, let alone a RAID array is abysmal for 
> streaming writes. Even a single drive should be capable of at least 
> 100MB/s.
>
> Here is the same statistic from one of my BPC v3 servers:
> dr:/mnt/imagestore# dd if=/dev/zero of=/var/backups/test.img bs=1M 
> count=1
> 1+0 records in
> 1+0 records out
> 1048576 bytes (10 GB) copied, 16.135 s, 650 MB/s
>
Whoops, that was writing to the SSD for the root FS I was rather 
surprised at the high number, but didn't stop and think properly.
Please see the revised stat:
dr:/mnt/imagestore# dd if=/dev/zero of=test.img bs=1M count=1
1+0 records in
1+0 records out
1048576 bytes (10 GB) copied, 81.5089 s, 129 MB/s

However, note that this is happening while BackupPC_nightly is running, 
so there is plenty of other IO happening in the background. (in v3 there 
is a nightly process which checks every file in the pool to see if it is 
still needed or not).

> This is a LV sitting on a RAID5 array:
> md0 : active raid5 sde1[4] sdc1[3] sdd1[2] sdb1[0]
>   11720658432 blocks super 1.2 level 5, 512k chunk, algorithm 2 
> [4/4] []
>
> Which is using these drives:
> Model Family: Western Digital Red (AF)
> Device Model: WDC WD40EFRX-68WT0N0
> User Capacity:4,000,787,030,016 bytes [4.00 TB]
> Sector Sizes: 512 bytes logical, 4096 bytes physical
> Rotation Rate:5400 rpm
>
> I'm sure you said you had 7200rpm disks, so you should get even better 
> performance for both random r/w as well as streaming writes. Which 
> brings me back to my earlier concern that you are using a VM for 
> backuppc, it is sharing it's performance with other things, which 
> works very poorly when dealing with spinning disks (even a streaming 
> write like your example is mixed with other random I/O which means it 
> kills performance).
>
> Please diagnose and resolve the underlying performance issues, then 
> come back to BPC and see how it performs.
>
> Regards,
> Adam
>


-- 
Adam Goryachev Website Managers www.websitemanagers.com.au

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Les Mikesell]

On Wed, Jan 13, 2016 at 3:57 PM, Gandalf Corvotempesta
 wrote:
> 2016-01-13 22:14 GMT+01:00 Gandalf Corvotempesta
> :
>> Nothing is hanged. When this occurs, no transfer is happening via network
>> and both "rsync_bpc" processes are parsing tons of these:
>
> Seems to be a write delay. rsync doesn't send new files until BPC has
> finished writes to disks.
>
> BTW, by changing compressione level from 3 to 1, seems to be a little faster.

If you aren't seeing system calls that haven't completed the process
may be CPU bound in user space - and a speedup from less compression
would make sense in that case.  But, that amount of time seems
extreme.

-- 
   Les Mikesell
 lesmikes...@gmail.com

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Adam Goryachev]

On 12/01/2016 19:00, Gandalf Corvotempesta wrote:
> 2016-01-12 8:29 GMT+01:00 Patrick Begou :
>> I'm using BackupPC 3.3.1 for a while now and backing up a partition of more 
>> than
>> 100GB  (from a macbook client) was requiering more than 24 hours on a 
>> backupPC
>> server with only 2GB of RAM. Increasing the RAM to 6GB has totally solved the
>> problem. Now a full backup on 180GB requires a little bit more than 3 hours.
> I'm doing this right now.
> I'll remove the whole /pc/ directory, add ram and start again.
Don't remove the pc directory itself, only the directories under it. 
Otherwise, you probably need to manually re-create it, and ensure 
permissions and ownership is correct.
> What do you think, should I also remove the pool directory and start
> totally clean ?
I would keep the pool in an effort to reduce the time it will take for 
the first backup I don't think it should be corrupted or causing any 
problem.

Regards,
Adam

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Adam Goryachev]

On 12/01/16 07:56, Gandalf Corvotempesta wrote:

Hi Gandalf,

Just jumping in here to hopefully provide some guidance. I understand 
that it can be extremely frustrating when you are migrating from a 
product you have used for years and has worked well enough to some new 
product and it isn't working as well as you had expected. The important 
points there are that your existing product has probably been configured 
and tuned appropriately to work well in your environment, while the new 
product hasn't had that opportunity yet, in addition, you know the older 
product well, while the new one is a big black box that is frustrating 
to work out what is wrong, why, and how to fix it. You sound very 
experienced and proficient (based on the types of debug you are doing), 
so I'm not belittling you or your knowledge at all, but equally, we are 
not your typical level 1 tech support droids either, lets work together.

So, please try to do the following:
1) Define and control your environment
* Define the specs of both your server and client (disks, ram, cpu, 
network, raid level, lvm, filesystem, etc)
* Only add one server at a time, when it is working well, add a second, 
eventually you will be comfortable with adding a batch of servers, but 
start controlled

2) When you come across a problem, try to provide as much detailed 
information as possible. This can include unmodified log files 
(excluding passwords), or similar. Don't just provide a tiny snippet or 
a summary, provide all the detail (eg, not just one or two lines with 
the error message, but the 50 lines before and 50 lines after).

3) Focus on one problem at a time
Often solving the first problem will also solve the other 5 unrelated 
issues you thought you had. BackupPC is a complex system, and it can 
take some time and effort to get it working smoothly.

4) IMHO, get yourself a development/testing/spare server to 
install/setup BackupPC. Leave your existing Bacula server in-place and 
working (so that you have actual working backups) and take some time to 
get BackupPC working.

See more comments below
> 2016-01-11 21:32 GMT+01:00 Les Mikesell :
>> I don't recognize that 'unknown host' log entry.  If the transfers
>> wait for whatever is writing it, it might cause a delay where the time
>> will depend on the DNS response - if you get an immediate NXDOMAIN
>> from a local server it should be quick but if you are referred to an
>> upstream and possibly firewalled server that won't respond you would
>> have a multiple-second wait for a timeout.
> dns resolution is done just on connect, not for each file.
> So, the delay would be just for the first connection attempt, after that,
> speed is normal as no dns resolution is made.
This might be true, but backuppc also needs to translate every log 
message it sees from every tool. If it doesn't understand that a 
specific message can be safely ignored, then it will treat it as an 
error and mark the backup as failed. I don't think a reverse DNS lookup 
would cause this message to be logged, at least not on the backuppc side 
of things. Where did you collect this log from? How many times is the 
error logged? What else does that log include or say?
> And, as wrote multiple times, the same server with standard rsync is working
> perfect. Only BPC is having trouble.
Sure, comparing standard rsync and BackupPC using rsync might be useful 
up to a certain point, but remember they do very different things, 
therefore performance will be different. If you are having a performance 
issue, then you will need to work out which component is causing that, 
and then decide what, if anything, you can do to try to resolve it. 
Performance issues are normally one of these:
1) Disk (not raw throughput, but random I/O). Use iostat while a backup 
is running to see what is happening.
2) RAM (a lot of ram can be used as a disk cache, not enough ram will 
massively increase your disk I/O even without swapping)
3) CPU (for compression, if it is enabled, and also rsync does lots of 
calculations/etc)
4) Bandwidth
5) Latency (both disk and network)

I've tried to order the above list in the common BPC performance 
problems. Run through the list, check both BPC server and client for 
each thing. When you find the problem, you can discuss it here, there 
could be some config option that will help, or it might require 
installing more/different hardware (eg more ram).

PS, can you explain the reason you are looking to move away from Bacula? 
What issue is it that you are trying to solve?

Regards,
Adam

-- 
Adam Goryachev Website Managers www.websitemanagers.com.au

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup 

Re: [BackupPC-users] BPC 4 very slow

[Les Mikesell]

On Mon, Jan 11, 2016 at 6:43 PM, Adam Goryachev
 wrote:
>
> 2016-01-10 00:21:40 Aborting backup up after signal INT
>
> This line seems to be out of context. I don't think it is related to the
> previous (completed) backup, and it shouldn't be related to the next
> backup that hasn't started yet. Did you do something here?

I was going to comment on that, but Adam is in a much better position
to help since he has run v4.   Is there any chance you could have
started more than one instance of the backuppc server?

-- 
  Les Mikesell
lesmikes...@gmail.com

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-12 1:08 GMT+01:00 Les Mikesell :
> That seems wrong - (maybe, depending on the counts you are configured
> to save).  But I thought in your first posting you said you had a
> 'partial'.  Those would be discarded when a better one completes.
> And one of your log entries mentioned a fatal error.

This is the BPC log for my first server (the one that actually is in
very very very slow copying phase from #2 to #1)

2016-01-08 19:26:48 Created directory /var/backups/backuppc/pc/x/refCnt
2016-01-08 19:26:48 full backup started for directory full
2016-01-09 21:07:03 full backup 0 complete, 4181687 files, 4181687
bytes, 1 xferErrs (0 bad files, 0 bad shares, 1 other)
2016-01-10 00:21:40 Aborting backup up after signal INT
2016-01-10 08:10:48 incr backup started for directory full
2016-01-10 10:27:51 Got fatal error during xfer (rsync_bpc exited with
benign status 24 (6144))
2016-01-10 19:23:47 full backup started for directory full
2016-01-11 18:16:02 full backup 1 complete, 4090748 files, 4090748
bytes, 24 xferErrs (0 bad files, 0 bad shares, 24 other)

First full dump was completed at 2016-01-09 21:07:03
A new incremental was started at 2016-01-10 08:10:48
This incremental was aborted due to an error 24. This should never
abort a backup, error 24 is benign and could happen in each backup if
spool files and dir are backupped (like php session files)

But, why on 2016-01-10 19:23:47 a new full was started? It should
start a new incremental. By the way, this new full lasted for 23 hours
as you can see (bytes counter is wrong, I don't have 4090748 bytes
but almost 150GB)

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Adam Goryachev]

On 12/01/16 11:26, Gandalf Corvotempesta wrote:
> 2016-01-12 1:08 GMT+01:00 Les Mikesell :
>> That seems wrong - (maybe, depending on the counts you are configured
>> to save).  But I thought in your first posting you said you had a
>> 'partial'.  Those would be discarded when a better one completes.
>> And one of your log entries mentioned a fatal error.
> This is the BPC log for my first server (the one that actually is in
> very very very slow copying phase from #2 to #1)
>
> 2016-01-08 19:26:48 Created directory /var/backups/backuppc/pc/x/refCnt
> 2016-01-08 19:26:48 full backup started for directory full
> 2016-01-09 21:07:03 full backup 0 complete, 4181687 files, 4181687
> bytes, 1 xferErrs (0 bad files, 0 bad shares, 1 other)
> 2016-01-10 00:21:40 Aborting backup up after signal INT
> 2016-01-10 08:10:48 incr backup started for directory full
> 2016-01-10 10:27:51 Got fatal error during xfer (rsync_bpc exited with
> benign status 24 (6144))
> 2016-01-10 19:23:47 full backup started for directory full
> 2016-01-11 18:16:02 full backup 1 complete, 4090748 files, 4090748
> bytes, 24 xferErrs (0 bad files, 0 bad shares, 24 other)
>
> First full dump was completed at 2016-01-09 21:07:03
> A new incremental was started at 2016-01-10 08:10:48
> This incremental was aborted due to an error 24. This should never
> abort a backup, error 24 is benign and could happen in each backup if
> spool files and dir are backupped (like php session files)
>
> But, why on 2016-01-10 19:23:47 a new full was started? It should
> start a new incremental. By the way, this new full lasted for 23 hours
> as you can see (bytes counter is wrong, I don't have 4090748 bytes
> but almost 150GB)
I'm confused and concerned about the above log...

2016-01-08 19:26:48 Created directory /var/backups/backuppc/pc/x/refCnt
2016-01-08 19:26:48 full backup started for directory full
2016-01-09 21:07:03 full backup 0 complete, 4181687 files, 4181687
bytes, 1 xferErrs (0 bad files, 0 bad shares, 1 other)

OK, so we created a new directory for the new host, and started the 
first backup (should be number 0).
The backup completed with 4181687 files (looks like a bug because bytes 
equals # files) however, it says there was 1 xferErrs. So we don't know 
if this caused BPC to mark the backup as incomplete or not.

2016-01-10 00:21:40 Aborting backup up after signal INT

This line seems to be out of context. I don't think it is related to the 
previous (completed) backup, and it shouldn't be related to the next 
backup that hasn't started yet. Did you do something here?

2016-01-10 08:10:48 incr backup started for directory full
2016-01-10 10:27:51 Got fatal error during xfer (rsync_bpc exited with
benign status 24 (6144))

Started a incremental backup, given the time, I'm guessing this was 
started manually, certainly the default doesn't start backups with that 
sort of offset, especially if it is the only server configured. How did 
you start this backup? Maybe from the web interface?
In any case, it has failed. Can you provide the actual backup log so we 
can see more details about why it failed?

2016-01-10 19:23:47 full backup started for directory full
2016-01-11 18:16:02 full backup 1 complete, 4090748 files, 4090748
bytes, 24 xferErrs (0 bad files, 0 bad shares, 24 other)

A full can certainly happen after a failed incremental, we don't know 
why. Again, the time looks very strange, how was this initiated? Can you 
provide copies of your configs? The detailed backup logs?

Regards,
Adam

-- 
Adam Goryachev Website Managers www.websitemanagers.com.au

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-11 23:29 GMT+01:00 Adam Goryachev :
> 1) Define and control your environment
> * Define the specs of both your server and client (disks, ram, cpu,
> network, raid level, lvm, filesystem, etc)
> * Only add one server at a time, when it is working well, add a second,
> eventually you will be comfortable with adding a batch of servers, but
> start controlled

server: DELL PE2950 with 2GB ram and 1 quad-core CPU, 6 SATA disks in RAID-5
client: Xen DomU, 8GB RAM, SAS disks
network: full gigabit (1 switch between server and client)

iperf show nearly about 1gbit

Initially I've added just 1 server: the biggest one, to test.
After first successful backup (lasted 2 days) i've added a smaller
server to check
if a smaller server would be parsed quickly.

With no apparent reason, BPC started a new backup for the firstly
added server, as incremental.
Then, first full and first incremental are suddenly disappeared and
now another full is running.

In the same time, the first full for the second server is still running.

Actually I have:
server1: type=backup, status = "copy #2 -> #1", count = 353170, start
time = 1/11 22:42
server2: type=full, status = "backup full", count = 1224681, start
time = 1/10 12:16

For server1 (Backup Summary table)

#1, Type=full, Filled=no, Level=0, Start Date=1/10 19:23,
Duration/mins=1372.2, Age/days=1.2, Server Backup
Path=/var/backups/backuppc/pc/x/1
#2, Type=full, Filled=yes, Level=0, Start Date=1/10 19:23,
Duration/mins=1372.2, Age/days=1.2, Server Backup
Path=/var/backups/backuppc/pc/x/2





> 2) When you come across a problem, try to provide as much detailed
> information as possible. This can include unmodified log files
> (excluding passwords), or similar. Don't just provide a tiny snippet or
> a summary, provide all the detail (eg, not just one or two lines with
> the error message, but the 50 lines before and 50 lines after).

I've never posted modified log. I've just removed the server names.

> 3) Focus on one problem at a time
> Often solving the first problem will also solve the other 5 unrelated
> issues you thought you had. BackupPC is a complex system, and it can
> take some time and effort to get it working smoothly.

I have just 1 problem, the other are related to this one (i think)

> 4) IMHO, get yourself a development/testing/spare server to
> install/setup BackupPC. Leave your existing Bacula server in-place and
> working (so that you have actual working backups) and take some time to
> get BackupPC working.

Bacula is stil running. Having a test environment is impossible at this time.

> This might be true, but backuppc also needs to translate every log
> message it sees from every tool. If it doesn't understand that a
> specific message can be safely ignored, then it will treat it as an
> error and mark the backup as failed. I don't think a reverse DNS lookup
> would cause this message to be logged, at least not on the backuppc side
> of things. Where did you collect this log from? How many times is the
> error logged? What else does that log include or say?

Posted log was from client, not from BPC. It was the rsync log.

> 1) Disk (not raw throughput, but random I/O). Use iostat while a backup
> is running to see what is happening.

Ok. i'll try tomorrow

> 2) RAM (a lot of ram can be used as a disk cache, not enough ram will
> massively increase your disk I/O even without swapping)

I would like to add ram, but backup are still running. I would like to
see if these
running process would terminate in the near future before rebooting.

> 3) CPU (for compression, if it is enabled, and also rsync does lots of
> calculations/etc)

Compression level 3 (the same used in Bacula)

> 4) Bandwidth

full gigabit, tested many times.

> 5) Latency (both disk and network)

network latency is good. Disk latency I've never checked.

> PS, can you explain the reason you are looking to move away from Bacula?
> What issue is it that you are trying to solve?

Too complicated. Many times it hangs with no apparent reason then it start again
after a couple of days. really, is a mess of software. I hate it.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-12 1:43 GMT+01:00 Adam Goryachev :
> OK, so we created a new directory for the new host, and started the
> first backup (should be number 0).

Yes, it was #0

> The backup completed with 4181687 files (looks like a bug because bytes
> equals # files) however, it says there was 1 xferErrs. So we don't know
> if this caused BPC to mark the backup as incomplete or not.

Exactly. This server has more or less 150GB of files.

> This line seems to be out of context. I don't think it is related to the
> previous (completed) backup, and it shouldn't be related to the next
> backup that hasn't started yet. Did you do something here?

Probably i've pressed "Stop/dequeue" from the admin panel to skip running
new backups for a while (it was just a test)

> Started a incremental backup, given the time, I'm guessing this was
> started manually, certainly the default doesn't start backups with that
> sort of offset, especially if it is the only server configured. How did
> you start this backup? Maybe from the web interface?

Absolutely nothis. BPC started this incremental on their own.

> In any case, it has failed. Can you provide the actual backup log so we
> can see more details about why it failed?

How can I get? I've posted the only log that I have in control panel.
Do you whant the error log ? It's huge and full of sensitive data

> 2016-01-10 19:23:47 full backup started for directory full
> 2016-01-11 18:16:02 full backup 1 complete, 4090748 files, 4090748
> bytes, 24 xferErrs (0 bad files, 0 bad shares, 24 other)
>
> A full can certainly happen after a failed incremental, we don't know
> why. Again, the time looks very strange, how was this initiated? Can you
> provide copies of your configs? The detailed backup logs?

I've never manually initiated a backup. All automatically by BPC.

Full configuration: http://pastebin.com/vU3Na2tP

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-12 1:53 GMT+01:00 Les Mikesell :
> Is there any chance you could have
> started more than one instance of the backuppc server?

Absolutely not. Only "rsync_bpc" is always twice for each running
backup (in this case, 1 running backup, so 2 rsync_bpc"

backuppc 27940  0.0  0.2  58776  5260 ?SJan08   1:20
/usr/bin/perl /usr/local/BackupPC/bin/BackupPC -d
backuppc 32144  0.0  0.5  59380 10960 ?SJan11   0:00
/usr/bin/perl /usr/local/BackupPC/bin/BackupPC_dump srv1
backuppc 32147 12.4 15.8 383596 326712 ?   D<   Jan11  23:11
/usr/bin/perl /usr/local/BackupPC/bin/BackupPC_backupDuplicate -h srv1
backuppc 32237  0.1  0.7  60680 14672 ?S00:46   0:05
/usr/bin/perl /usr/local/BackupPC/bin/BackupPC_dump srv2
backuppc 32241  0.8  8.6 200376 177196 ?   S00:46   0:31
/usr/local/bin/rsync_bpc --bpc-top-dir /var/backups/backuppc
--bpc-host-name srv2 --bpc-share-name full --bpc-bkup-num 0
--bpc-bkup-comp 3 --bpc-bkup-prevnum -1 --bpc-bkup-prevcomp -1
--bpc-bkup-inode0 1 --bpc-log-level 1 --super --recursive
--protect-args --numeric-ids --perms --owner --group -D --times
--links --hard-links --delete --partial --log-format=log: %o %i %B
%8U,%8G %9l %f%L --stats --checksum
--password-file=/var/backups/backuppc/pc/srv2/.rsyncdpw32237
--exclude=/proc --exclude=/sys --exclude=tmp/ --exclude=/var/cache
--exclude=/var/log/lastlog --exclude=/var/log/rsync*
--exclude=/var/lib/mlocate --exclude=/var/spool --exclude=/media
--exclude=/mnt x@srv2::full /
backuppc 32242  0.5  5.5 195348 114016 ?   D00:46   0:18
/usr/local/bin/rsync_bpc --bpc-top-dir /var/backups/backuppc
--bpc-host-name srv2 --bpc-share-name full --bpc-bkup-num 0
--bpc-bkup-comp 3 --bpc-bkup-prevnum -1 --bpc-bkup-prevcomp -1
--bpc-bkup-inode0 1 --bpc-log-level 1 --super --recursive
--protect-args --numeric-ids --perms --owner --group -D --times
--links --hard-links --delete --partial --log-format=log: %o %i %B
%8U,%8G %9l %f%L --stats --checksum
--password-file=/var/backups/backuppc/pc/srv2/.rsyncdpw32237
--exclude=/proc --exclude=/sys --exclude=tmp/ --exclude=/var/cache
--exclude=/var/log/lastlog --exclude=/var/log/rsync*
--exclude=/var/lib/mlocate --exclude=/var/spool --exclude=/media
--exclude=/mnt x@srv2::full /

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Les Mikesell]

On Mon, Jan 11, 2016 at 5:00 PM, Gandalf Corvotempesta
 wrote:
> >
> server: DELL PE2950 with 2GB ram and 1 quad-core CPU, 6 SATA disks in RAID-5

RAID-5 will cost a lot in performance on small writes.

> Initially I've added just 1 server: the biggest one, to test.
> After first successful backup (lasted 2 days) i've added a smaller
> server to check
> if a smaller server would be parsed quickly.
>
> With no apparent reason, BPC started a new backup for the firstly
> added server, as incremental.

I think if your full took 2 days and daily backups are scheduled,
starting an incremental would be expected.

> Then, first full and first incremental are suddenly disappeared and
> now another full is running.

That seems wrong - (maybe, depending on the counts you are configured
to save).  But I thought in your first posting you said you had a
'partial'.  Those would be discarded when a better one completes.
And one of your log entries mentioned a fatal error.

-- 
  Les Mikesell
lesmikes...@gmail.com

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Adam Goryachev]

Please don't trim so much, it is very useful to still keep the original 
log that is being discussed.

On 12/01/16 11:57, Gandalf Corvotempesta wrote:
> 2016-01-12 1:43 GMT+01:00 Adam Goryachev 
> :
>> The backup completed with 4181687 files (looks like a bug because bytes
>> equals # files) however, it says there was 1 xferErrs. So we don't know
>> if this caused BPC to mark the backup as incomplete or not.
> Exactly. This server has more or less 150GB of files.
However, we still don't know about the error.


>> 2016-01-10 00:21:40 Aborting backup up after signal INT
>>
>> This line seems to be out of context. I don't think it is related to the
>> previous (completed) backup, and it shouldn't be related to the next
>> backup that hasn't started yet. Did you do something here?
> Probably i've pressed "Stop/dequeue" from the admin panel to skip running
> new backups for a while (it was just a test)
I've pasted the log entry back in, and I don't see that error being 
logged from simply pressing stop/dequeue while a backup is not running. 
I would expect to see that error if stopping a running backup.

Just as a test, I click stop/dequeue on one of my own BPC v4 servers, 
and asked it to prevent any backups for one hour. There was no entry 
logged at all for this.

I would strongly suggest that something has happened here which could be 
explained by trying lots of random things when trying to get it working. 
I'd probably suggest to stop BPC, delete the pc/server1 directory, and 
then start BPC, and re-run the first/initial backup. It might be quicker 
than the real first backup since a lot of the files will still be in the 
pool.
>> Started a incremental backup, given the time, I'm guessing this was
>> started manually, certainly the default doesn't start backups with that
>> sort of offset, especially if it is the only server configured. How did
>> you start this backup? Maybe from the web interface?
> Absolutely nothis. BPC started this incremental on their own.
OK, well, just looks strange though technically, it shouldn't matter 
how the backups are started, as long as you don't use the CLI tools that 
are not meant to be used directly.
>
>> In any case, it has failed. Can you provide the actual backup log so we
>> can see more details about why it failed?
> How can I get? I've posted the only log that I have in control panel.
> Do you whant the error log ? It's huge and full of sensitive data
Yes, the error log or the xferlog. It should only contain 
filenames/directory names, hopefully that is not so sensitive data? At 
best, it should contain a lot more detail from any errors encountered. 
We don't need (or want) the full log, that could be 100's of MB, but a 
reasonable snippet to clearly show what was happening before/after the 
relevant errors. At least the first 20 lines and last 20 lines are 
usually reasonably useful.

I'm referring to the log file for the specific backup So click on 
the host, you see the table of backups, and underneath is a table of 
Xfer Error Summary which provides links to the XferLOG and Errors.
>> 2016-01-10 19:23:47 full backup started for directory full
>> 2016-01-11 18:16:02 full backup 1 complete, 4090748 files, 4090748
>> bytes, 24 xferErrs (0 bad files, 0 bad shares, 24 other)
>>
>> A full can certainly happen after a failed incremental, we don't know
>> why. Again, the time looks very strange, how was this initiated? Can you
>> provide copies of your configs? The detailed backup logs?
> I've never manually initiated a backup. All automatically by BPC.
>
> Full configuration: http://pastebin.com/vU3Na2tP

No idea please post any logs or configs inline in the email. Not 
only does that allow people to look at them in the future (ie, the 
archives in 2 years time) but also it lets everyone see them without 
having to go and refer to another website.

Skipping lines that are blank/commented is usually a good idea. Both the 
global config and the host specific config would be useful. Obviously, 
obfuscate username/password/etc as needed, but the more you edit the 
file the harder it can be to work out what you have obfuscated. eg, it 
can be better to change the username jamesw to malcom rather than 
changing it to x.

Regards,
Adam

-- 
Adam Goryachev Website Managers www.websitemanagers.com.au

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-12 1:43 GMT+01:00 Adam Goryachev :

> A full can certainly happen after a failed incremental, we don't know
> why. Again, the time looks very strange, how was this initiated? Can you
> provide copies of your configs? The detailed backup logs?

I can post this, for the incremental: http://pastebin.com/raw/v74rGTFq

There is no backup #0 here, only #1 and #2, both identical (BPC is
copying #2 in #1).

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Sorin Srbu]

> -Original Message-
> From: Gandalf Corvotempesta [mailto:gandalf.corvotempe...@gmail.com]
> Sent: den 11 januari 2016 09:27
> To: General list for user discussion, questions and support
> Subject: Re: [BackupPC-users] BPC 4 very slow
>
> 2016-01-11 6:57 GMT+01:00 Alexander Moisseev <mois...@mezonplus.ru>:
> > I believe weekDays should be in range 0...6 (0 is Sunday and 6 is 
> > Saturday), i.e.
> > weekDays  => [0, 1, 2, 3, 4, 5, 6],
>
> You are right, my mistake.
> But for the rest of my questions ?

Please excuse a side-track.

Aren't there usually quite some questions about how to set up the 
blackout-periods and related?
There is a lot of confusion on how to set this up.

May I suggest an overhaul in the GUI, and may be the docs description, for 
this feature to simplify things?


FWIW, in Sweden Saturdays and Sundays are considered weekend days.
Weeksdays (=workweek) are Mondays through Fridays.

-- 
//Sorin


smime.p7s
Description: S/MIME cryptographic signature
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-12 2:19 GMT+01:00 Adam Goryachev :
> Please don't trim so much, it is very useful to still keep the original
> log that is being discussed.

I've not trimmed at all.

> I would strongly suggest that something has happened here which could be
> explained by trying lots of random things when trying to get it working.
> I'd probably suggest to stop BPC, delete the pc/server1 directory, and
> then start BPC, and re-run the first/initial backup. It might be quicker
> than the real first backup since a lot of the files will still be in the
> pool.

Ok, i'll try after adding more ram.

> Yes, the error log or the xferlog. It should only contain
> filenames/directory names, hopefully that is not so sensitive data? At
> best, it should contain a lot more detail from any errors encountered.
> We don't need (or want) the full log, that could be 100's of MB, but a
> reasonable snippet to clearly show what was happening before/after the
> relevant errors. At least the first 20 lines and last 20 lines are
> usually reasonably useful.

Are sensitive data bacause i'm backupping some file/folder in format:

"username@host" so I can't show that on a public mailing list.

> Skipping lines that are blank/commented is usually a good idea. Both the
> global config and the host specific config would be useful. Obviously,
> obfuscate username/password/etc as needed, but the more you edit the
> file the harder it can be to work out what you have obfuscated. eg, it
> can be better to change the username jamesw to malcom rather than
> changing it to x.

I've posted the whole config, not trimmed at all.
I don't have host specific config except for a single line with rsync password.
What I've posted is *exactly* what i'm using, no trim or customization was made
except replacing server hostname with 'srv1', 'srv2' and so on and
users with 'x'

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Patrick Begou]

Gandalf Corvotempesta wrote:
> Exactly. This server has more or less 150GB of files. 

I'm using BackupPC 3.3.1 for a while now and backing up a partition of more 
than 
100GB  (from a macbook client) was requiering more than 24 hours on a backupPC 
server with only 2GB of RAM. Increasing the RAM to 6GB has totally solved the 
problem. Now a full backup on 180GB requires a little bit more than 3 hours.

May be you should first increase the RAM of your server.

Patrick

-- 
===
|  Equipe M.O.S.T. |  |
|  Patrick BEGOU   | mailto:patrick.be...@grenoble-inp.fr |
|  LEGI|  |
|  BP 53 X | Tel 04 76 82 51 35   |
|  38041 GRENOBLE CEDEX| Fax 04 76 82 52 71   |
===


--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-11 21:32 GMT+01:00 Les Mikesell :
> I don't recognize that 'unknown host' log entry.  If the transfers
> wait for whatever is writing it, it might cause a delay where the time
> will depend on the DNS response - if you get an immediate NXDOMAIN
> from a local server it should be quick but if you are referred to an
> upstream and possibly firewalled server that won't respond you would
> have a multiple-second wait for a timeout.

dns resolution is done just on connect, not for each file.
So, the delay would be just for the first connection attempt, after that,
speed is normal as no dns resolution is made.

And, as wrote multiple times, the same server with standard rsync is working
perfect. Only BPC is having trouble.

> You could test the theory with "nslookup 172.17.0.1" to see how long
> the response takes if that is the actual address.

$ time nslookup 172.17.0.1
Server: 127.0.1.1
Address: 127.0.1.1#53

** server can't find 1.0.17.172.in-addr.arpa: NXDOMAIN


real 0m0.108s
user 0m0.004s
sys 0m0.004s

is fast enough for you?

DNS is not an issue here.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Les Mikesell]

On Mon, Jan 11, 2016 at 1:31 PM, Gandalf Corvotempesta
 wrote:
> 2016-01-11 19:10 GMT+01:00 Les Mikesell :
>> Wild guess here, but 'host unknown' usually means something has done a
>> DNS lookup (or reverse, number to name) that has failed.  DNS lookups
>> can be slow.Maybe sticking the client hosts and IPs in your
>> /etc/hosts file would help if your reverse DNS doesn't work.
>
> I don't have reverse set for local machines, and as wrote before, the same
> configuration is used with plain rsync without issue at all.
>
> If DNS resolution is slow (but this is not the case, i don't see any
> delay), the same
> delay should even happen with standard rsync.

I don't recognize that 'unknown host' log entry.  If the transfers
wait for whatever is writing it, it might cause a delay where the time
will depend on the DNS response - if you get an immediate NXDOMAIN
from a local server it should be quick but if you are referred to an
upstream and possibly firewalled server that won't respond you would
have a multiple-second wait for a timeout.

You could test the theory with "nslookup 172.17.0.1" to see how long
the response takes if that is the actual address.


-- 
   Les Mikesell
 lesmikes...@gmail.com

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Alexander Moisseev]

On 11.01.16 11:33, Sorin Srbu wrote:
> Aren't there usually quite some questions about how to set up the
> blackout-periods and related?
> There is a lot of confusion on how to set this up.
>
> May I suggest an overhaul in the GUI, and may be the docs description, for
> this feature to simplify things?
>

The weekDays numbering is documented here: 
http://backuppc.sourceforge.net/BackupPC-4.0.0alpha3_doc.html#What-to-backup-and-when-to-do-it

You suggestion is reasonable, but any changes in the code or documentation 
isn't possible at the moment. You can find details in the BackupPC-devel 
mailing list latest threads.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Sorin Srbu]

> -Original Message-
> From: Alexander Moisseev [mailto:mois...@mezonplus.ru]
> Sent: den 11 januari 2016 12:48
> To: backuppc-users@lists.sourceforge.net
> Subject: Re: [BackupPC-users] BPC 4 very slow
> 
> On 11.01.16 11:33, Sorin Srbu wrote:
> > Aren't there usually quite some questions about how to set up the
> > blackout-periods and related?
> > There is a lot of confusion on how to set this up.
> >
> > May I suggest an overhaul in the GUI, and may be the docs description,
for
> > this feature to simplify things?
> >
> 
> The weekDays numbering is documented here:
> http://backuppc.sourceforge.net/BackupPC-4.0.0alpha3_doc.html#What-to-
> backup-and-when-to-do-it

I know about it, I remember this being a pretty darn high threshold to pass
when I first started with BPC, and I have a feeling I might've irritated the
list pretty good with my questions at the time.

> You suggestion is reasonable, but any changes in the code or documentation
> isn't possible at the moment. You can find details in the BackupPC-devel
> mailing list latest threads.

Ah, thanks.

Another solution ocurred to me; how about a companion app in the form of a
web-gui on the docs site with a helper to set up the blackout periods, you
know - a simple thing with checkboxes ("I want the backup to occur at these
times" etc, click "prepare code" and you'll get a piece of code you can copy
and paste into the config-file directly.

It might be a bit over the top though...

-- 
//Sorin



smime.p7s
Description: S/MIME cryptographic signature
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Alexander Moisseev]

On 11.01.16 11:27, Gandalf Corvotempesta wrote:
> But for the rest of my questions ?
>

> Is the partial backup a "good one"? Should't it be an incremental ?
> Why both are "filled"? If I understood properly, only the last one is
> filled.

2016-01-10 08:10:48 incr backup started for directory full
2016-01-10 10:27:51 Got fatal error during xfer (rsync_bpc exited with
benign status 24 (6144))

The incremental backup was interrupted due to fatal error. So, it is "partial". 
It means only some files were backed up. 
http://backuppc.sourceforge.net/BackupPC-4.0.0alpha3_doc.html#Backup-basics

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-11 12:37 GMT+01:00 Alexander Moisseev :
> The incremental backup was interrupted due to fatal error. So, it is 
> "partial". It means only some files were backed up. 
> http://backuppc.sourceforge.net/BackupPC-4.0.0alpha3_doc.html#Backup-basics

Status 24 should not considered an error (or at least, a configuration
variable should be added to enable or disable the error 24 detection).
On many server, vanished files are normal (for example, PHP session
file and so on).
Having a "failed job" for an error 24 is not good, because the job was
executed properly, just some file was not copied, as it should be with
volatile file.

By the way, i've added a second server yesterday, and the first full
backup is still running, as per image:
http://s12.postimg.org/4enfzc7u5/screen.jpg

First line: i don't know what is trying to do. First full was
completed with about 1.400.000 files. Second backup was the same. This
is the third backup, why is trying to parse 3.358.721 files ?
Second line: this server was added yesterday and seems to be stuck in
first full backup. No files are parsed from many many hours (rsync log
on client doesn't log anything from this morning). strace to xfer pid
is showing some files parsed.

Yesterday I had 2 backups for server in first line, 1 full, 1
incremental. Now I have just 1 "active" backup. No more full or
incremental.


Something is not working properly and, after all, is absolutely
nonsense 24/48 hours for each backup. The same server with Bacula was
completed in 16 hours.

How can I troubleshoot this ?

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/