Hi list, I'm new here. At first, thank you for BackupPC! :)
I'm trying to backup a new client. The problem is: rsync never starts to transfer files, not even after 12 hours of waiting. rsync is doing something, though. More on that below. The client is a quad core 2.8 GHz CPU, 8 GB RAM and 1.6 TB of many many small files in a RAID0. CPUs 75-95% idle most of the time, load around 0.3. No swap used. rsync 3.0.7 on the client, rsync 3.0.6 on the server. Negotiated protocol 28 (hmm). BackupPC server is backing up to a 8 TB iSCSI RAID5 drive. So, when I initiate the full backup manually, and then attach with strace to the rsync process on the BackupPC server I see activity, but it's so slow. Roughly it takes about 5 seconds for 10 lines of read/select/write. On the client I can see via lsof that rsync is going through the filesystem and the many mini-files. On the client rsync consumes like 0.x-1% CPU and the "virt" mem size is growing slowly (80M after 10 minutes, in the beginning 40M; that's what "top" says). Load on the client goes from 0.3 to about 1.8 a few minutes after I start the full backup. But CPUs stay at around 70-90% idle. Last evening I started the backup at 22:00 and in the morning at around 09:00 the customer killed the rsync procress manually because he said the client was becoming unresponsive. Even at 09:00 there was not yet a single file transferred. I'm guessing the client is slowing down after so many hours because rsync has used up all memory as it is caching the list of files to be transferred? Unfortunately I didn't monitor yet what the rsync process looks like after 12 hours, but will do. I'm now starting the rsync process with "ionice -c 3 nice -n10 /usr/bin/rsync" because the customer complained, but if the memory usage is the reason for the client to slow down, this will probably not help. What I don't understand yet is - why does rsync on the client tell rsync on the server about the files it is currently going through? I mean, it is not even transferring them yet! I see these select/read/write outputs in strace attached to the rsync server process, and as it appears this means "Hello, I'm the client, and I'm telling you which files I go through now on the client filesystem". But why does the rsync server process even need to know about that at this point in time? I'm guessing it can't be the comparing mechanism of BackupPC, because there are no files to compare yet! There are 0 backups with 0 files. I'm backing up about 25 other clients just fine, but this is the only one with such a large amount of files and data. This is why I suspect the issue is with the many files/big data. All other clients have 150 GB maximum, after backuping. These are kinda slow too, but at least they already start tranferring files after a couple of seconds, minutes or 1, 2 hours at most. Lots of guessing ... Any suggestions on what I could do or what could go wrong here? Thank you very much! Markus ------------------------------------------------------------------------------ Monitor your physical, virtual and cloud infrastructure from a single web console. Get in-depth insight into apps, servers, databases, vmware, SAP, cloud infrastructure, etc. Download 30-day Free Trial. Pricing starts from $795 for 25 servers or applications! http://p.sf.net/sfu/zoho_dev2dev_nov _______________________________________________ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
