Re: [BackupPC-users] BackupPC_tarCreate failed

[Carl Wilhelm Soderstrom]

On 01/13 07:54 , David Rotger wrote:
> because when I try to restore a localhost backup I get this error!?!?
> 
> BackupPC_tarCreate failed

Can you paste the command you're using when you get this error?
Are there any other messages?
Just copy and paste the sequence of commands leading up to this, and any
resulting output.


-- 
Carl Soderstrom
Systems Administrator
Real-Time Enterprises
www.real-time.com

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

[BackupPC-users] Garbage into XferLog.z

[Bzzzz]

Hi list,

First, I already have had this problem for this particular machine when
I wanted it to be a client (garbage into NewFileList).
It runs under a Debian sid 64bits, just like half of my park
(1/2: sid 64bits & 32bits, 2/2: jessie 32bits).

Today, it becomes the backuppc svr and I still have problems
(NewFileList is now empty).

The main problem is:
/usr/share/backuppc/bin/BackupPC_zcat pc/backuppcsvr.domain/XferLOG.z
only return garbage (a bit like if file had a different encoding,
which is not possible as all machines works with UTF-8); so,
no backup is made from the server itself:(

* Where could it come from?

* How to fix that?

Jean-Yves

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BackupPC_tarCreate failed

[Carl Wilhelm Soderstrom]

On 01/13 08:51 , David Rotger wrote:
> I use the web interface, when I restore a TAR localhost bakcup, in the
> status section I see:
> 
> localhost  restore
> root 2016-01-13
> 20:00 2016-01-13 19:53
> BackupPC_tarCreate failed

Ok, now's the time to learn to use the command line tools to troubleshoot
problems with BackupPC. The BackupPC_tarCreate command is actually this:

backuppc@archivist-2:~$ /usr/share/backuppc/bin/BackupPC_tarCreate --help
/usr/share/backuppc/bin/BackupPC_tarCreate version [unknown] calling
Getopt::Std::getopts (version 1.07 [paranoid]),
running under Perl version 5.18.2.

Usage: BackupPC_tarCreate [-OPTIONS [-MORE_OPTIONS]] [--] [PROGRAM_ARG1 ...]

The following single-character options are accepted:
With arguments: -e -h -n -p -r -s -b -w
Boolean (without arguments): -L -l -t

Options may be merged together.  -- stops processing of options.
Space is not required between options and their arguments.
  [Now continuing due to backward compatibility and excessive paranoia.
   See 'perldoc Getopt::Std' about $Getopt::Std::STANDARD_HELP_VERSION.]
usage: /usr/share/backuppc/bin/BackupPC_tarCreate [options]
files/directories...
  Required options:
 -h host host from which the tar archive is created
 -n dumpNum  dump number from which the tar archive is created
 A negative number means relative to the end (eg -1
 means the most recent dump, -2 2nd most recent etc).
 -s shareNameshare name from which the tar archive is created

  Other options:
 -t  print summary totals
 -r pathRemove   path prefix that will be replaced with pathAdd
 -p pathAdd  new path prefix
 -b BLOCKS   output write buffer size in 512-byte blocks (default
20; same as tar)
 -w readBufSzbuffer size for reading files (default 1048576 = 1MB)
 -e charset  charset for encoding file names (default: value of
 $Conf{ClientCharset} when backup was done)
 -l  just print a file listing; don't generate an archive
 -L  just print a detailed file listing; don't generate an
archive


So an actual command would look something like this below, where:
-h localhost == the host to restore from
-n -1 == the most recent backup (or you could look at the list of backup
numbers in the web interface to find the backup from a particular date)
-s / == the share to restore from (in the case of a Windows box it might be
'cDrive' or something like that)
/ == the files to restore. in this case, all the files from the root
directory onward

/usr/share/backuppc/bin/BackupPC_tarCreate -h localhost -n -1 -s / /

Note however that this will dump the tar stream straight to STDOUT (i.e.
your terminal). So you'll have to redirect that or stream it elsewhere.
Here's an example of creating a tarball in your current directory (make sure
there's enough space first, using 'df -h').

/usr/share/backuppc/bin/BackupPC_tarCreate \
-h localhost -n -1 -s / / >  localhost_restore-2016-01-13.tar

I stuck the '\' in there just because your mail client may wrap lines, and I
wanted to be clear that this is all one line of command.

Or if you'd like to move the backup across the network, you can pipe the
output of BackupPC_tarCreate through netcat and move it to some other
machine without having to put it on the local disk first.

/usr/share/backuppc/bin/BackupPC_tarCreate \
-h localhost -n -1 -s / / | nc host.example.com  

On the receiving side, you'd have a netcat process listening and piping the
output to tar, which in the example below will dump the files out onto the
filesystem in the current directory.

nc -l  | tar xvpf - 


Experiment with all these a bit, and see what errors you get when you try to
run a really simple restore command like this:

/usr/share/backuppc/bin/BackupPC_tarCreate \
-h localhost -n -1 -s / /etc > /dev/null

To be clear, you don't have to break the above into two lines, and you don't
need the '\' if you don't break it into two lines, that's just for clarity
since I don't know how your mail client will format that line. Also the
redirection to /dev/null is just so your screen and filesystem don't get
filled up while you figure out what may be wrong. If you like you can
redirect to a file.

Sorry if some of this is very low-level, I don't know what your skill level
is.

-- 
Carl Soderstrom
Systems Administrator
Real-Time Enterprises
www.real-time.com

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___

[BackupPC-users] Creating a "read everywhere" user to backup Windows profiles

[Andreas Piening]

I wonder what the easiest / best way is to create a „read everywhere“ user on 
ms windows to create backups with via CIFS / SMBFS.

Ideally I would like to run a short .cmd script or do a couple of clicks to 
give a local windows user (let’s assume ‚backuppc‘) full read access to 
everything under c:\Users. Even better with write access to be able to restore 
in place.
I know that I can enable inheritance for permissions in c:\Users and overwrite 
all permissions on subfolders with the current one. But this would also enable 
read for everyone for every user on other users profiles which I don’t like. 
And even this does not work everywhere, even not with an administrative 
account. I need to take ownership recursively in order to do that and I don’t 
want to own other users files.

Is there a better way? 
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-13 21:07 GMT+01:00 Gandalf Corvotempesta
:
> 2016/01/13 20:48:21 [19453] 2016/01/13 20:48:21: host unknown
> (172.17.0.1) send
> home/user/domains/domain.it/public_html/cgi-bin/.htaccess (17 bytes).
> Total 43 bytes.
> 2016/01/13 20:48:21 [19453] 2016/01/13 20:48:21: host unknown
> (172.17.0.1) send
> home/user/domains/domain.it/.htpasswd/.protected.list (0 bytes). Total
> 39 bytes.
> 2016/01/13 21:03:29 [19453] 2016/01/13 21:03:29: host unknown
> (172.17.0.1) send home/user/domains/domain.it/awstats/.htaccess (233
> bytes). Total 276 bytes.
> 2016/01/13 21:03:29 [19453] 2016/01/13 21:03:29: host unknown
> (172.17.0.1) send
> home/user/domains/domain.it/awstats/awstats.domain.it.1003.alldomains.html
> (5934 bytes). Total 5977 bytes.
>
> Totally frozen between 2016/01/13 20:48:21 and 2016/01/13 21:03:29

Raw performance by direct rsync between these two servers:

receiving incremental file list
test.img
  1,073,741,824 100%   25.04MB/s0:00:40 (xfr#1, to-chk=0/1)

sent 77 bytes  received 1,073,873,061 bytes  24,686,738.80 bytes/sec
total size is 1,073,741,824  speedup is 1.00


24.5MB/s, not too much, but not too bad. 24 times faster than BPC
(with BPC i got about 1MB/s)



raw performance writing to disk on BPC server (same partitions used by
BPC as storage):

# dd if=/dev/zero of=/var/backups/test.img bs=1M count=1
^C8815+0 records in
8815+0 records out
9243197440 bytes (9.2 GB) copied, 142.267 s, 65.0 MB/s



So, network is not an issue. "plain" rsync is not an issue, write
performance to disks are not an issue..
The only remaining is BPC.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

[BackupPC-users] BackupPC_tarCreate failed

[David Rotger]

Hello,

because when I try to restore a localhost backup I get this error!?!?

BackupPC_tarCreate failed

Anyone can help me?!!?

thanks!


-- 
David Rotger
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-12 8:29 GMT+01:00 Patrick Begou :
> I'm using BackupPC 3.3.1 for a while now and backing up a partition of more 
> than
> 100GB  (from a macbook client) was requiering more than 24 hours on a backupPC
> server with only 2GB of RAM. Increasing the RAM to 6GB has totally solved the
> problem. Now a full backup on 180GB requires a little bit more than 3 hours.

Ok, let's start again from the beginning:

# cat /etc/debian_version
8.2

# grep MemTotal /proc/meminfo
MemTotal:8190512 kB

# grep 'model name' /proc/cpuinfo  | head -n1
model name : Intel(R) Xeon(R) CPU   E5410  @ 2.33GHz

# mount | grep backups
/dev/mapper/vg0-lv_backups on /var/backups type ext3
(rw,nosuid,nodev,noexec,noatime,nodiratime,nobarrier)

Click on "start full backup" in admin page

# cat /var/log/BackupPC/LOG
2016-01-13 20:21:26 Running BackupPC_Admin_SCGI (pid=29415)
2016-01-13 20:22:11 Reading hosts file
2016-01-13 20:22:11 BackupPC started, pid 29463
2016-01-13 20:22:11 Running BackupPC_Admin_SCGI (pid=29464)
2016-01-13 20:22:11 Next wakeup is 2016-01-13 23:00:00
2016-01-13 20:28:58 User backuppc requested backup of srv1 (srv1)
2016-01-13 20:28:58 Started full backup on srv1 (pid=29473, share=full)


Let's wait

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BackupPC_tarCreate failed

[Andreas Piening]

Please take a look at Server > LOG file in the web interface. The bottom lines 
should give you some more information on what exactly caused the failure.

> Am 13.01.2016 um 20:51 schrieb David Rotger :
> 
> Hi Carl,
> 
> I use the web interface, when I restore a TAR localhost bakcup, in the status 
> section I see:
> 
> localhost restore 
> root <> 2016-01-13 20:002016-01-13 19:53
> BackupPC_tarCreate failed 

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BackupPC_tarCreate failed

[Les Mikesell]

On Wed, Jan 13, 2016 at 1:51 PM, David Rotger  wrote:

> Hi Carl,
>
> I use the web interface, when I restore a TAR localhost bakcup, in the
> status section I see:
>
> localhost  restore
> root 2016-01-13 20:00 2016-01-13 19:53
> BackupPC_tarCreate failed
>
>
Are you using ssh to connect as root to localhost like you would another
client or have you set up some other mechanism to get the appropriate
privileges?

-- 
   Les Mikesell
 lesmikes...@gmail.com
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-13 20:39 GMT+01:00 Gandalf Corvotempesta
:
> # cat /var/log/BackupPC/LOG
> 2016-01-13 20:21:26 Running BackupPC_Admin_SCGI (pid=29415)
> 2016-01-13 20:22:11 Reading hosts file
> 2016-01-13 20:22:11 BackupPC started, pid 29463
> 2016-01-13 20:22:11 Running BackupPC_Admin_SCGI (pid=29464)
> 2016-01-13 20:22:11 Next wakeup is 2016-01-13 23:00:00
> 2016-01-13 20:28:58 User backuppc requested backup of srv1 (srv1)
> 2016-01-13 20:28:58 Started full backup on srv1 (pid=29473, share=full)

Same issues. This is "rsync.log" client side:

2016/01/13 20:48:21 [19453] 2016/01/13 20:48:21: host unknown
(172.17.0.1) send
home/user/domains/domain.it/public_html/cgi-bin/.htaccess (17 bytes).
Total 43 bytes.
2016/01/13 20:48:21 [19453] 2016/01/13 20:48:21: host unknown
(172.17.0.1) send
home/user/domains/domain.it/.htpasswd/.protected.list (0 bytes). Total
39 bytes.
2016/01/13 21:03:29 [19453] 2016/01/13 21:03:29: host unknown
(172.17.0.1) send home/user/domains/domain.it/awstats/.htaccess (233
bytes). Total 276 bytes.
2016/01/13 21:03:29 [19453] 2016/01/13 21:03:29: host unknown
(172.17.0.1) send
home/user/domains/domain.it/awstats/awstats.domain.it.1003.alldomains.html
(5934 bytes). Total 5977 bytes.

Totally frozen between 2016/01/13 20:48:21 and 2016/01/13 21:03:29

BPC server side was writing to pool (I've seen many activity by using
"strace" to rsync_bpc)

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Les Mikesell]

On Wed, Jan 13, 2016 at 2:52 PM, Gandalf Corvotempesta
 wrote:
> >>
>> Totally frozen between 2016/01/13 20:48:21 and 2016/01/13 21:03:29

Did your strace test show a hanging system call on any of the active
processes in this time?

> raw performance writing to disk on BPC server (same partitions used by
> BPC as storage):
>
> # dd if=/dev/zero of=/var/backups/test.img bs=1M count=1
> ^C8815+0 records in
> 8815+0 records out
> 9243197440 bytes (9.2 GB) copied, 142.267 s, 65.0 MB/s

That's not at all like what rsync would be doing when it merges
changes to a compressed file.

-- 
   Les Mikesell
  lesmikes...@gmail.com

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-13 22:07 GMT+01:00 Les Mikesell :
> Did your strace test show a hanging system call on any of the active
> processes in this time?

Nothing is hanged. When this occurs, no transfer is happening via network
and both "rsync_bpc" processes are parsing tons of these:


FIRST PROCES:
read(5, "82f3016ca8f4b309aa141fb1aee9dfb0"..., 8184) = 4117
select(6, [5], [], NULL, {60, 0})   = 1 (in [5], left {59, 855296})
read(5, "34166a01128e65c0e98ce2a634ea"..., 8184) = 4093
select(6, [5], [], NULL, {60, 0})   = 1 (in [5], left {59, 988318})

SECOND PROCESS:
select(4, [3], [], NULL, {60, 0})   = 1 (in [3], left {59, 34})
read(3, "63db4c25f333ca\0\202\3Vg9\211\247\346\"N\233<\215\320x\4\272"...,
4092) = 2896
select(4, [3], [], NULL, {60, 0})   = 1 (in [3], left {59, 999675})
read(3, 
"197addc7b4\0001\2V\v\v\222\2655\242\213\242\310P\222\343\331\211Q\244T\237"...,
1196) = 1196
select(7, NULL, [6], [6], {60, 0})  = 1 (out [6], left {59, 99})
write(6, "690f3f5dc38571ac1d63db4c25f333ca"..., 4092) = 4092
select(7, NULL, [6], [6], {60, 0})  = 1 (out [6], left {59, 98})
write(6, "\363a\306v}\376\364@\241\236{:A ", 14) = 14
select(4, [3], [], NULL, {60, 0})   = 1 (in [3], left {59, 906237})
read(3, "\374\17\0\7", 4)   = 4
select(4, [3], [], NULL, {60, 0})   = 1 (in [3], left {59, 999806})
read(3, "ff5cd88ec882380bd1ade93a3c24\0M\2V"..., 4092) = 2896


What's the meaning of two rsync_bpc processes?


> That's not at all like what rsync would be doing when it merges
> changes to a compressed file.

I know, but having a slow disk would slow down also rsync and bpc.
This test told me that disks are working properly and bottleneck shold
be somewhere else.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BackupPC_tarCreate failed

[David Rotger]

Hi Carl,

I use the web interface, when I restore a TAR localhost bakcup, in the
status section I see:

localhost  restore
root 2016-01-13
20:00 2016-01-13 19:53
BackupPC_tarCreate failed

2016-01-13 20:43 GMT+01:00 Carl Wilhelm Soderstrom :

> On 01/13 07:54 , David Rotger wrote:
> > because when I try to restore a localhost backup I get this error!?!?
> >
> > BackupPC_tarCreate failed
>
> Can you paste the command you're using when you get this error?
> Are there any other messages?
> Just copy and paste the sequence of commands leading up to this, and any
> resulting output.
>
>
> --
> Carl Soderstrom
> Systems Administrator
> Real-Time Enterprises
> www.real-time.com
>
>
> --
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
> ___
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
>



-- 
David Rotger
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BackupPC_tarCreate failed

[David Rotger]

when I type the command they exit to the help menú...

$ ./BackupPC_tarCreate -h localhost -n -4 -s /mnt/smb >
/tmp/localhost_restore.tar
usage: ./BackupPC_tarCreate [options] files/directories...
  Required options:
 -h host host from which the tar archive is created
 -n dumpNum  dump number from which the tar archive is created
 A negative number means relative to the end (eg -1
 means the most recent dump, -2 2nd most recent etc).
 -s shareNameshare name from which the tar archive is created

  Other options:
 -t  print summary totals
 -r pathRemove   path prefix that will be replaced with pathAdd
 -p pathAdd  new path prefix
 -b BLOCKS   output write buffer size in 512-byte blocks (default
20; same as tar)
 -w readBufSzbuffer size for reading files (default 1048576 = 1MB)
 -e charset  charset for encoding file names (default: value of
 $Conf{ClientCharset} when backup was done)
 -l  just print a file listing; don't generate an archive
 -L  just print a detailed file listing; don't generate an
archive
$

2016-01-14 0:08 GMT+01:00 Adam Goryachev <
mailingli...@websitemanagers.com.au>:

> On 14/01/16 09:58, David Rotger wrote:
> > And the command with backuppc user says:
> >
> > $ ./BackupPC_tarCreate -h localhost -n -4 -s /mnt/smb >
> > localhost_restore.tar
> > -su: 10: cannot create localhost_restore.tar: Permission denied
> >
>
> Try this:
> $ ./BackupPC_tarCreate -h localhost -n -4 -s /mnt/smb >
> /tmp/localhost_restore.tar
>
> You should have permission to write in /tmp/ but make sure you have
> enough space there first:
> df -H /tmp
>
> Regards,
> Adam
>
> --
> Adam Goryachev
> Website Managers
> P: +61 2 8304 a...@websitemanagers.com.au
> F: +61 2 8304 0001 www.websitemanagers.com.au
>
>
> --
> Adam Goryachev Website Managers www.websitemanagers.com.au
>
>
> --
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
> ___
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
>



-- 
David Rotger
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Adam Goryachev]

On 14/01/16 08:14, Gandalf Corvotempesta wrote:
> 2016-01-13 22:07 GMT+01:00 Les Mikesell :
>> Did your strace test show a hanging system call on any of the active
>> processes in this time?
> Nothing is hanged. When this occurs, no transfer is happening via network
> and both "rsync_bpc" processes are parsing tons of these:
>
>
> FIRST PROCES:
> read(5, "82f3016ca8f4b309aa141fb1aee9dfb0"..., 8184) = 4117
> select(6, [5], [], NULL, {60, 0})   = 1 (in [5], left {59, 855296})
> read(5, "34166a01128e65c0e98ce2a634ea"..., 8184) = 4093
> select(6, [5], [], NULL, {60, 0})   = 1 (in [5], left {59, 988318})
>
> SECOND PROCESS:
> select(4, [3], [], NULL, {60, 0})   = 1 (in [3], left {59, 34})
> read(3, "63db4c25f333ca\0\202\3Vg9\211\247\346\"N\233<\215\320x\4\272"...,
> 4092) = 2896
> select(4, [3], [], NULL, {60, 0})   = 1 (in [3], left {59, 999675})
> read(3, 
> "197addc7b4\0001\2V\v\v\222\2655\242\213\242\310P\222\343\331\211Q\244T\237"...,
> 1196) = 1196
> select(7, NULL, [6], [6], {60, 0})  = 1 (out [6], left {59, 99})
> write(6, "690f3f5dc38571ac1d63db4c25f333ca"..., 4092) = 4092
> select(7, NULL, [6], [6], {60, 0})  = 1 (out [6], left {59, 98})
> write(6, "\363a\306v}\376\364@\241\236{:A ", 14) = 14
> select(4, [3], [], NULL, {60, 0})   = 1 (in [3], left {59, 906237})
> read(3, "\374\17\0\7", 4)   = 4
> select(4, [3], [], NULL, {60, 0})   = 1 (in [3], left {59, 999806})
> read(3, "ff5cd88ec882380bd1ade93a3c24\0M\2V"..., 4092) = 2896
>
>
> What's the meaning of two rsync_bpc processes?
>
>
>> That's not at all like what rsync would be doing when it merges
>> changes to a compressed file.
> I know, but having a slow disk would slow down also rsync and bpc.
> This test told me that disks are working properly and bottleneck shold
> be somewhere else.
>

Ummm, really? I think you are confused. Depending on where exactly the 
above processes are reading or writing to (most likely it isn't network, 
which means it is almost certainly your backuppc server disks) will tell 
you where the bottleneck is. You have identified that the client is 
providing the data to the server quickly enough, but the server is too 
slow to process this data (ie, do whatever needs to happen to save it in 
the correct place). This is almost certainly one or more of the 
following reasons:
1) Slow I/O
2) Not enough RAM leading to not enough cache leading to slow I/O
3) Slow CPU

You provided some other information in another email:
> Raw performance by direct rsync between these two servers:
>
> receiving incremental file list
> test.img
>1,073,741,824 100%   25.04MB/s0:00:40 (xfr#1, to-chk=0/1)
>
> sent 77 bytes  received 1,073,873,061 bytes  24,686,738.80 bytes/sec
> total size is 1,073,741,824  speedup is 1.00
>
>
> 24.5MB/s, not too much, but not too bad. 24 times faster than BPC
> (with BPC i got about 1MB/s)
This is completely rubbish, it isn't a useful comparison of anything. I 
am almost certain that your actual client isn't made up of files with a 
average size of 1GB. In fact the snippet of the rsyncd log that you 
previously provided showed very small files. It still isn't a meaningful 
comparison, but at least it is more realistic if you used the actual 
files you are trying to backup, even if it is only a subset of them.

BTW, the reason it isn't so relevant is because backuppc does a lot more 
work on the server side than plain rsync, the client side performance is 
relevant, and could at least show that the client is capable.
> raw performance writing to disk on BPC server (same partitions used by
> BPC as storage):
>
> # dd if=/dev/zero of=/var/backups/test.img bs=1M count=1
> ^C8815+0 records in
> 8815+0 records out
> 9243197440 bytes (9.2 GB) copied, 142.267 s, 65.0 MB/s
Totally irrelevant. BackupPC is doing lots of small random reads and 
writes. However, maybe that is relevant, because 65MB/s on any single 
HDD from the past 5 years, let alone a RAID array is abysmal for 
streaming writes. Even a single drive should be capable of at least 100MB/s.

Here is the same statistic from one of my BPC v3 servers:
dr:/mnt/imagestore# dd if=/dev/zero of=/var/backups/test.img bs=1M 
count=1
1+0 records in
1+0 records out
1048576 bytes (10 GB) copied, 16.135 s, 650 MB/s

This is a LV sitting on a RAID5 array:
md0 : active raid5 sde1[4] sdc1[3] sdd1[2] sdb1[0]
   11720658432 blocks super 1.2 level 5, 512k chunk, algorithm 2 
[4/4] []

Which is using these drives:
Model Family: Western Digital Red (AF)
Device Model: WDC WD40EFRX-68WT0N0
User Capacity:4,000,787,030,016 bytes [4.00 TB]
Sector Sizes: 512 bytes logical, 4096 bytes physical
Rotation Rate:5400 rpm

I'm sure you said you had 7200rpm disks, so you should get even better 
performance for both random r/w as well as streaming writes. Which 
brings me back to my earlier concern that you are using a VM for 
backuppc, it is sharing it's 

Re: [BackupPC-users] BackupPC_tarCreate failed

[David Rotger]

And the command with backuppc user says:

$ ./BackupPC_tarCreate -h localhost -n -4 -s /mnt/smb >
localhost_restore.tar
-su: 10: cannot create localhost_restore.tar: Permission denied


2016-01-13 23:45 GMT+01:00 David Rotger :

> Yes, if I do ' su - backuppc ' the command work. Now in the host
> configuration for localhost I put the ' backuppc ' user. But when I try to
> restore, the error comes again...
>
>
>
> 2016-01-13 23:37 GMT+01:00 Carl Wilhelm Soderstrom :
>
>> On 01/13 11:17 , David Rotger wrote:
>> > root@RFSERVER2:/usr/share/backuppc/bin# ./BackupPC_tarCreate
>> > ./BackupPC_tarCreate: Wrong user: my userid is 0, instead of 111
>> (backuppc)
>> > Please su backuppc first
>> > BackupPC::Lib->new failed
>> > root@RFSERVER2:/usr/share/backuppc/bin#
>>
>> Sorry, I meant to mention that (as the error explains above) you need to
>> be
>> the 'backuppc' user instead of 'root' for most of the commands to work.
>>
>> So do 'su - backuppc', or if that fails, do 'su - backuppc -s /bin/bash'
>> (presuming that /bin/bash is the path to your bash executable file).
>>
>> --
>> Carl Soderstrom
>> Systems Administrator
>> Real-Time Enterprises
>> www.real-time.com
>>
>>
>> --
>> Site24x7 APM Insight: Get Deep Visibility into Application Performance
>> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
>> Monitor end-to-end web transactions and take corrective actions now
>> Troubleshoot faster and improve end-user experience. Signup Now!
>> http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
>> ___
>> BackupPC-users mailing list
>> BackupPC-users@lists.sourceforge.net
>> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
>> Wiki:http://backuppc.wiki.sourceforge.net
>> Project: http://backuppc.sourceforge.net/
>>
>
>
>
> --
> David Rotger
>



-- 
David Rotger
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BackupPC_tarCreate failed

[David Rotger]

The log file says:

2016-01-13 19:53:46 User backuppc requested restore to localhost
 (localhost
)
2016-01-13 19:53:46 Started restore on localhost
 (pid=10366)
2016-01-13 19:53:50 Restore failed on localhost
 (BackupPC_tarCreate
failed)

And the command:


root@RFSERVER2:/usr/share/backuppc/bin# ./BackupPC_tarCreate
./BackupPC_tarCreate: Wrong user: my userid is 0, instead of 111 (backuppc)
Please su backuppc first
BackupPC::Lib->new failed
root@RFSERVER2:/usr/share/backuppc/bin#



2016-01-13 21:27 GMT+01:00 Carl Wilhelm Soderstrom :

> On 01/13 08:51 , David Rotger wrote:
> > I use the web interface, when I restore a TAR localhost bakcup, in the
> > status section I see:
> >
> > localhost  restore
> > root 2016-01-13
> > 20:00 2016-01-13 19:53
> > BackupPC_tarCreate failed
>
> Ok, now's the time to learn to use the command line tools to troubleshoot
> problems with BackupPC. The BackupPC_tarCreate command is actually this:
>
> backuppc@archivist-2:~$ /usr/share/backuppc/bin/BackupPC_tarCreate --help
> /usr/share/backuppc/bin/BackupPC_tarCreate version [unknown] calling
> Getopt::Std::getopts (version 1.07 [paranoid]),
> running under Perl version 5.18.2.
>
> Usage: BackupPC_tarCreate [-OPTIONS [-MORE_OPTIONS]] [--] [PROGRAM_ARG1
> ...]
>
> The following single-character options are accepted:
> With arguments: -e -h -n -p -r -s -b -w
> Boolean (without arguments): -L -l -t
>
> Options may be merged together.  -- stops processing of options.
> Space is not required between options and their arguments.
>   [Now continuing due to backward compatibility and excessive paranoia.
>See 'perldoc Getopt::Std' about $Getopt::Std::STANDARD_HELP_VERSION.]
> usage: /usr/share/backuppc/bin/BackupPC_tarCreate [options]
> files/directories...
>   Required options:
>  -h host host from which the tar archive is created
>  -n dumpNum  dump number from which the tar archive is created
>  A negative number means relative to the end (eg -1
>  means the most recent dump, -2 2nd most recent etc).
>  -s shareNameshare name from which the tar archive is created
>
>   Other options:
>  -t  print summary totals
>  -r pathRemove   path prefix that will be replaced with pathAdd
>  -p pathAdd  new path prefix
>  -b BLOCKS   output write buffer size in 512-byte blocks (default
> 20; same as tar)
>  -w readBufSzbuffer size for reading files (default 1048576 = 1MB)
>  -e charset  charset for encoding file names (default: value of
>  $Conf{ClientCharset} when backup was done)
>  -l  just print a file listing; don't generate an archive
>  -L  just print a detailed file listing; don't generate an
> archive
>
>
> So an actual command would look something like this below, where:
> -h localhost == the host to restore from
> -n -1 == the most recent backup (or you could look at the list of backup
> numbers in the web interface to find the backup from a particular date)
> -s / == the share to restore from (in the case of a Windows box it might be
> 'cDrive' or something like that)
> / == the files to restore. in this case, all the files from the root
> directory onward
>
> /usr/share/backuppc/bin/BackupPC_tarCreate -h localhost -n -1 -s / /
>
> Note however that this will dump the tar stream straight to STDOUT (i.e.
> your terminal). So you'll have to redirect that or stream it elsewhere.
> Here's an example of creating a tarball in your current directory (make
> sure
> there's enough space first, using 'df -h').
>
> /usr/share/backuppc/bin/BackupPC_tarCreate \
> -h localhost -n -1 -s / / >  localhost_restore-2016-01-13.tar
>
> I stuck the '\' in there just because your mail client may wrap lines, and
> I
> wanted to be clear that this is all one line of command.
>
> Or if you'd like to move the backup across the network, you can pipe the
> output of BackupPC_tarCreate through netcat and move it to some other
> machine without having to put it on the local disk first.
>
> /usr/share/backuppc/bin/BackupPC_tarCreate \
> -h localhost -n -1 -s / / | nc host.example.com 
>
> On the receiving side, you'd have a netcat process listening and piping the
> output to tar, which in the example below will dump the files out onto the
> filesystem in the current directory.
>
> nc -l  | tar xvpf -
>
>
> Experiment with all these a bit, and see what errors you get when you try
> to
> run a really simple restore command like this:
>
> /usr/share/backuppc/bin/BackupPC_tarCreate \
> -h localhost -n -1 -s / /etc > /dev/null
>
> To be clear, you don't have to break the above into two lines, and 

Re: [BackupPC-users] BackupPC_tarCreate failed

[Adam Goryachev]

On 14/01/16 09:37, Carl Wilhelm Soderstrom wrote:
> On 01/13 11:17 , David Rotger wrote:
>> root@RFSERVER2:/usr/share/backuppc/bin# ./BackupPC_tarCreate
>> ./BackupPC_tarCreate: Wrong user: my userid is 0, instead of 111 (backuppc)
>> Please su backuppc first
>> BackupPC::Lib->new failed
>> root@RFSERVER2:/usr/share/backuppc/bin#
> Sorry, I meant to mention that (as the error explains above) you need to be
> the 'backuppc' user instead of 'root' for most of the commands to work.
>
> So do 'su - backuppc', or if that fails, do 'su - backuppc -s /bin/bash'
> (presuming that /bin/bash is the path to your bash executable file).
>
Or login to the webpage, navigate to the localhost and below the list of 
backups you should have the Restore Summary. Click the "Restore #" link, 
and it should show the detailed log of what happened. Please post that 
full log.

Regards,
Adam

-- 
Adam Goryachev Website Managers www.websitemanagers.com.au

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BackupPC_tarCreate failed

[Adam Goryachev]

On 14/01/16 09:58, David Rotger wrote:
> And the command with backuppc user says:
>
> $ ./BackupPC_tarCreate -h localhost -n -4 -s /mnt/smb >
> localhost_restore.tar
> -su: 10: cannot create localhost_restore.tar: Permission denied
>

Try this:
$ ./BackupPC_tarCreate -h localhost -n -4 -s /mnt/smb > 
/tmp/localhost_restore.tar

You should have permission to write in /tmp/ but make sure you have 
enough space there first:
df -H /tmp

Regards,
Adam

-- 
Adam Goryachev
Website Managers
P: +61 2 8304 a...@websitemanagers.com.au
F: +61 2 8304 0001 www.websitemanagers.com.au


-- 
Adam Goryachev Website Managers www.websitemanagers.com.au

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-13 22:14 GMT+01:00 Gandalf Corvotempesta
:
> Nothing is hanged. When this occurs, no transfer is happening via network
> and both "rsync_bpc" processes are parsing tons of these:

Seems to be a write delay. rsync doesn't send new files until BPC has
finished writes to disks.

BTW, by changing compressione level from 3 to 1, seems to be a little faster.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Garbage into XferLog.z

[Adam Goryachev]

On 14/01/16 07:22, B wrote:
> Hi list,
>
> First, I already have had this problem for this particular machine when
> I wanted it to be a client (garbage into NewFileList).
> It runs under a Debian sid 64bits, just like half of my park
> (1/2: sid 64bits & 32bits, 2/2: jessie 32bits).
>
> Today, it becomes the backuppc svr and I still have problems
> (NewFileList is now empty).
>
> The main problem is:
> /usr/share/backuppc/bin/BackupPC_zcat pc/backuppcsvr.domain/XferLOG.z
> only return garbage (a bit like if file had a different encoding,
> which is not possible as all machines works with UTF-8); so,
> no backup is made from the server itself:(
>
> * Where could it come from?
>
> * How to fix that?
>
Does this happen for all machines you are backing up, or just one?
Can you provide a copy of the "garbage" file (ie, attach the xferlog.z 
if it isn't too big).

Regards,
Adam
-- 
Adam Goryachev Website Managers www.websitemanagers.com.au

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-14 0:06 GMT+01:00 Adam Goryachev :
> However, note that this is happening while BackupPC_nightly is running,
> so there is plenty of other IO happening in the background. (in v3 there
> is a nightly process which checks every file in the pool to see if it is
> still needed or not).

There is no nightly at this time here. I'm running nightly in the
middle of the day :)

$Conf{WakeupSchedule} = [12, 23, 24, 1, 2, 3, 4, 5, 6, 7];

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Gandalf Corvotempesta]

2016-01-14 0:06 GMT+01:00 Adam Goryachev :
> Whoops, that was writing to the SSD for the root FS I was rather
> surprised at the high number, but didn't stop and think properly.
> Please see the revised stat:
> dr:/mnt/imagestore# dd if=/dev/zero of=test.img bs=1M count=1
> 1+0 records in
> 1+0 records out
> 1048576 bytes (10 GB) copied, 81.5089 s, 129 MB/s

This is mine:

# dd if=/dev/zero of=/var/backups/test.img bs=1M count=2000
2000+0 records in
2000+0 records out
2097152000 bytes (2.1 GB) copied, 15.4692 s, 136 MB/s


>> I'm sure you said you had 7200rpm disks, so you should get even better
>> performance for both random r/w as well as streaming writes. Which
>> brings me back to my earlier concern that you are using a VM for
>> backuppc, it is sharing it's performance with other things, which
>> works very poorly when dealing with spinning disks (even a streaming
>> write like your example is mixed with other random I/O which means it
>> kills performance).

I'm not using a VM (please don't make confusion between users), It's a
DELL PowerEdge 2950 with 8GB ram, hardware raid and 6x2TB @7200 disks
in RAID-5

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Adam Goryachev]

On 14/01/16 09:29, Adam Goryachev wrote:
> On 14/01/16 08:14, Gandalf Corvotempesta wrote:
>
>> raw performance writing to disk on BPC server (same partitions used by
>> BPC as storage):
>>
>> # dd if=/dev/zero of=/var/backups/test.img bs=1M count=1
>> ^C8815+0 records in
>> 8815+0 records out
>> 9243197440 bytes (9.2 GB) copied, 142.267 s, 65.0 MB/s
> Totally irrelevant. BackupPC is doing lots of small random reads and 
> writes. However, maybe that is relevant, because 65MB/s on any single 
> HDD from the past 5 years, let alone a RAID array is abysmal for 
> streaming writes. Even a single drive should be capable of at least 
> 100MB/s.
>
> Here is the same statistic from one of my BPC v3 servers:
> dr:/mnt/imagestore# dd if=/dev/zero of=/var/backups/test.img bs=1M 
> count=1
> 1+0 records in
> 1+0 records out
> 1048576 bytes (10 GB) copied, 16.135 s, 650 MB/s
>
Whoops, that was writing to the SSD for the root FS I was rather 
surprised at the high number, but didn't stop and think properly.
Please see the revised stat:
dr:/mnt/imagestore# dd if=/dev/zero of=test.img bs=1M count=1
1+0 records in
1+0 records out
1048576 bytes (10 GB) copied, 81.5089 s, 129 MB/s

However, note that this is happening while BackupPC_nightly is running, 
so there is plenty of other IO happening in the background. (in v3 there 
is a nightly process which checks every file in the pool to see if it is 
still needed or not).

> This is a LV sitting on a RAID5 array:
> md0 : active raid5 sde1[4] sdc1[3] sdd1[2] sdb1[0]
>   11720658432 blocks super 1.2 level 5, 512k chunk, algorithm 2 
> [4/4] []
>
> Which is using these drives:
> Model Family: Western Digital Red (AF)
> Device Model: WDC WD40EFRX-68WT0N0
> User Capacity:4,000,787,030,016 bytes [4.00 TB]
> Sector Sizes: 512 bytes logical, 4096 bytes physical
> Rotation Rate:5400 rpm
>
> I'm sure you said you had 7200rpm disks, so you should get even better 
> performance for both random r/w as well as streaming writes. Which 
> brings me back to my earlier concern that you are using a VM for 
> backuppc, it is sharing it's performance with other things, which 
> works very poorly when dealing with spinning disks (even a streaming 
> write like your example is mixed with other random I/O which means it 
> kills performance).
>
> Please diagnose and resolve the underlying performance issues, then 
> come back to BPC and see how it performs.
>
> Regards,
> Adam
>


-- 
Adam Goryachev Website Managers www.websitemanagers.com.au

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BackupPC_tarCreate failed

[Carl Wilhelm Soderstrom]

On 01/13 11:17 , David Rotger wrote:
> root@RFSERVER2:/usr/share/backuppc/bin# ./BackupPC_tarCreate
> ./BackupPC_tarCreate: Wrong user: my userid is 0, instead of 111 (backuppc)
> Please su backuppc first
> BackupPC::Lib->new failed
> root@RFSERVER2:/usr/share/backuppc/bin#

Sorry, I meant to mention that (as the error explains above) you need to be
the 'backuppc' user instead of 'root' for most of the commands to work.

So do 'su - backuppc', or if that fails, do 'su - backuppc -s /bin/bash'
(presuming that /bin/bash is the path to your bash executable file).

-- 
Carl Soderstrom
Systems Administrator
Real-Time Enterprises
www.real-time.com

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BackupPC_tarCreate failed

[David Rotger]

Yes, if I do ' su - backuppc ' the command work. Now in the host
configuration for localhost I put the ' backuppc ' user. But when I try to
restore, the error comes again...



2016-01-13 23:37 GMT+01:00 Carl Wilhelm Soderstrom :

> On 01/13 11:17 , David Rotger wrote:
> > root@RFSERVER2:/usr/share/backuppc/bin# ./BackupPC_tarCreate
> > ./BackupPC_tarCreate: Wrong user: my userid is 0, instead of 111
> (backuppc)
> > Please su backuppc first
> > BackupPC::Lib->new failed
> > root@RFSERVER2:/usr/share/backuppc/bin#
>
> Sorry, I meant to mention that (as the error explains above) you need to be
> the 'backuppc' user instead of 'root' for most of the commands to work.
>
> So do 'su - backuppc', or if that fails, do 'su - backuppc -s /bin/bash'
> (presuming that /bin/bash is the path to your bash executable file).
>
> --
> Carl Soderstrom
> Systems Administrator
> Real-Time Enterprises
> www.real-time.com
>
>
> --
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
> ___
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
>



-- 
David Rotger
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Creating a "read everywhere" user to backup Windows profiles

[Adam Goryachev]

On 14/01/16 07:11, Andreas Piening wrote:
> I wonder what the easiest / best way is to create a „read everywhere“ user on 
> ms windows to create backups with via CIFS / SMBFS.
>
> Ideally I would like to run a short .cmd script or do a couple of clicks to 
> give a local windows user (let’s assume ‚backuppc‘) full read access to 
> everything under c:\Users. Even better with write access to be able to 
> restore in place.
> I know that I can enable inheritance for permissions in c:\Users and 
> overwrite all permissions on subfolders with the current one. But this would 
> also enable read for everyone for every user on other users profiles which I 
> don’t like. And even this does not work everywhere, even not with an 
> administrative account. I need to take ownership recursively in order to do 
> that and I don’t want to own other users files.
>
> Is there a better way?

Isn't there a specific "Backup Operator" account on windows which has 
"super" permissions for exactly this reason? I'm not sure if that 
account will work over samba though?

Regards,
Adam

-- 
Adam Goryachev Website Managers www.websitemanagers.com.au

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BPC 4 very slow

[Les Mikesell]

On Wed, Jan 13, 2016 at 3:57 PM, Gandalf Corvotempesta
 wrote:
> 2016-01-13 22:14 GMT+01:00 Gandalf Corvotempesta
> :
>> Nothing is hanged. When this occurs, no transfer is happening via network
>> and both "rsync_bpc" processes are parsing tons of these:
>
> Seems to be a write delay. rsync doesn't send new files until BPC has
> finished writes to disks.
>
> BTW, by changing compressione level from 3 to 1, seems to be a little faster.

If you aren't seeing system calls that haven't completed the process
may be CPU bound in user space - and a speedup from less compression
would make sense in that case.  But, that amount of time seems
extreme.

-- 
   Les Mikesell
 lesmikes...@gmail.com

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/