Re: Multi-Threading?

2011-05-19 Thread CACook
Ok, you guys' suggestions helped alot. I changed --compress to --whole-file, and in fact that increased my speed from 4.5Mb/s to 18.5! Much better. Still core-bound though. (not disk, or network) Now that I've finished my restore, I've run iperf. Droog -- Hex 937 Mbits/sec! Merlin -- Hex

Re: Multi-Threading?

2011-05-17 Thread Matt McCutchen
On Tue, 2011-05-17 at 08:45 -0700, cac...@quantum-sci.com wrote: The connexion is Gb enet end-to-end, and is running at only 40Mb/s. It has far more capacity than that. The only limiting factor I can see is on the backup server one core of the CPU is running 100% rsync. Clearly rsync is not

Re: Multi-Threading?

2011-05-17 Thread CACook
Thanks, but the whole function of my backup server pivots on rsync features. Need something rsync-like, but multi-threaded, or with a whole lot less overhead. On Tuesday 17 May, 2011 09:18:01 Chris Hawkins wrote: I have no idea about potential rsync modifications, but you might try FDT to

Re: Multi-Threading?

2011-05-17 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Also, make sure you aren't running with compression as it will just waste CPU cycles. If you are running rsync over ssh it might also be worth while to setup a one time use rsyncd or even NFS mount to eliminate the encryption overhead. On 05/17/11

Re: Multi-Threading?

2011-05-17 Thread Carlos Carvalho
Matt McCutchen (m...@mattmccutchen.net) wrote on 17 May 2011 12:50: On Tue, 2011-05-17 at 08:45 -0700, cac...@quantum-sci.com wrote: The connexion is Gb enet end-to-end, and is running at only 40Mb/s. It has far more capacity than that. The only limiting factor I can see is on the backup

Re: Multi-Threading?

2011-05-17 Thread Chuck Wolber
On Tue, 17 May 2011, cac...@quantum-sci.com wrote: In researching this I find that a change to multi-threaded goodness would require a massive rewrite, and would only be considered for an rsync replacement. Abstracting the core functionality into a librsync.so would be really spiffy too...

Re: Multi-Threading?

2011-05-17 Thread CACook
On Tuesday 17 May, 2011 10:43:21 you wrote: Wow! That's a lot of data you're transferring into one server, even with a GB pipe you're using, which appears to be using a fraction of it. Yes, it's my home theater computer with all the movies. I've just set up a RAID array and need to move my

Re: Multi-Threading?

2011-05-17 Thread Matt McCutchen
On Tue, 2011-05-17 at 10:54 -0700, Chuck Wolber wrote: On Tue, 17 May 2011, cac...@quantum-sci.com wrote: In researching this I find that a change to multi-threaded goodness would require a massive rewrite, and would only be considered for an rsync replacement. Abstracting the core

Re: Multi-Threading?

2011-05-17 Thread Chuck Wolber
On Tue, 17 May 2011, Matt McCutchen wrote: On Tue, 2011-05-17 at 10:54 -0700, Chuck Wolber wrote: Abstracting the core functionality into a librsync.so would be really spiffy too... All easy to say, harder to do (and maintain). I'm thankful that rsync meets my needs right now, and