Re: rsync speedup - how ?

2009-08-07 Thread Daniel.Li
On Fri, 2009-08-07 at 18:44 +0200, devz...@web.de wrote: devz...@web.de wrote: so, instead of 500M i would transfer 100GB over the network. that`s no option. I don't see how you came up with such numbers. If files change completely then I don't see why you would transfer more (or

How to get first level folder names with rsync commands?

2009-06-30 Thread Daniel.Li
Dear List, I set up a NetBackup destination on my server. And I backup test1 to NetBackup/test1, test2 to NetBackup/test2. Now I wanna know how many folders that have been backuped. It should be 2 folders, with the name of test1 and test2. But I don't know if there is any way to do so? Any

How to first level folder names with rsync commands?

2009-06-30 Thread Daniel.Li
Dear List, I set up a NetBackup destination on my server. And I backup test1 to NetBackup/test1, test2 to NetBackup/test2. Now I wanna know how many folders that have been backuped. It should be 2 folders, with the name of test1 and test2. But I don't know if there is any way to do so? Any

RE: How to get first level folder names with rsync commands?

2009-06-30 Thread Daniel.Li
. (assuming something unixy, of course -- cygwin stuff might work) -Original Message- From: rsync-bounces+tony=servacorp@lists.samba.org [mailto:rsync-bounces+tony=servacorp@lists.samba.org] On Behalf Of Daniel.Li Sent: Wednesday, July 01, 2009 12:17 AM To: Rsync Subject

-acv option, what will be the default delete option?

2009-06-16 Thread Daniel.Li
Dear List, If I use -acv for rsync options, what will I choose as default delete option? --deletedelete extraneous files from destination dirs --delete-before receiver deletes before transfer, not during --delete-during receiver deletes during transfer (default)

Weird behavior in receive_data function

2009-06-10 Thread Daniel.Li
Dear List, I'm trying to get diff/removed data and it's offset out. So I write a functions in receive_data. When I run backup, I found there is a weird behavior which I don't understand. i = recv_token(f_in, data) will receive (i = -1, offset2 = 0) some where in the middle of the transfer

Re: Weird behavior in receive_data function

2009-06-10 Thread Daniel.Li
On Thu, 2009-06-11 at 07:11 +0800, Daniel.Li wrote: Dear List, I'm trying to get diff/removed data and it's offset out. So I write a functions in receive_data. When I run backup, I found there is a weird behavior which I don't understand. i = recv_token(f_in, data) will receive (i = -1

Key for high CPU usage

2009-06-10 Thread Daniel.Li
Dear List, I'm trying to take a closer look at rsync code, and found when we run daemon, it will take a lot of CPU (400Mhz). So I'm interested in Which part of rsync code on ver 3.0.5 consuming CPU a lot? Can anyone here help to lighten me up? So I can try to improve the performance or low the

Re: rsync --daemon. Can I open more than one instances?

2009-06-08 Thread Daniel.Li
On Mon, 2009-06-08 at 22:39 -0500, Rahul Nabar wrote: If you're not using different partitions on different disks, then doing anything in parallel is probably going to slow it down (reading/writing all over the disk, means more seektime, which means waiting). No, its just a single

what kind attrs are going to be transfered by rsync?

2009-06-04 Thread Daniel.Li
Dear list, I have met this error. I wanna know what attrs are transfer by rsync program? sent 5045 bytes received 262853 bytes 107159.20 bytes/sec total size is 130560 speedup is 0.49 rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1535)

Re: what kind attrs are going to be transfered by rsync?

2009-06-04 Thread Daniel.Li
-06-04 at 14:44 +0800, Daniel.Li wrote: Dear list, I have met this error. I wanna know what attrs are transfer by rsync program? sent 5045 bytes received 262853 bytes 107159.20 bytes/sec total size is 130560 speedup is 0.49 rsync error: some files/attrs were not transferred (see

Re: rsync --daemon. Can I open more than one instances?

2009-06-03 Thread Daniel.Li
On Wed, 2009-06-03 at 20:57 -0500, Rahul Nabar wrote: Is there a way to speed up rsync by opening more than one daemon in parallel. daemon will create a process when a connection is established. So the limit is hardware. I use rsync --daemon to start rsync. I was wondering if opening more

Which function in generator trigger the file transfer?

2009-06-02 Thread Daniel.Li
Dear List, I checked the function generate_files, which might be simply stand for generator, but didn't find where triggers the file transfer. As receiver will use read_ndx_and_attrs to read iflags to judge if it's need to be transfered? If I'm wrong, please correct me. Can anyone help to point

Re: Which function in generator trigger the file transfer?

2009-06-02 Thread Daniel.Li
. On Tue, 2009-06-02 at 15:31 +0800, Daniel.Li wrote: Dear List, I checked the function generate_files, which might be simply stand for generator, but didn't find where triggers the file transfer. As receiver will use read_ndx_and_attrs to read iflags to judge if it's need to be transfered

Re: How to emulate rdiff behaviour

2009-05-31 Thread Daniel.Li
Dear Wayne, Excellent, that's really what I have expected! Is it stable now? Cause I have found that this feature seems to be unstable before ver 3.0.6. - Fixed a --read-batch hang when rsync is reading a batch file that was created from an incremental-recursion transfer.

Re: How to emulate rdiff behaviour

2009-05-31 Thread Daniel.Li
On Sun, 2009-05-31 at 14:34 -0400, Matt McCutchen wrote: And one more thing here: If you are going to prepare this batch file, it seems there will be double the workload of network, see below statements? Is that right? rsync -av --only-write-batch=/batches/$DATE bhost:/backup2/

Re: How to emulate rdiff behaviour

2009-05-31 Thread Daniel.Li
On Sun, 2009-05-31 at 23:11 -0400, Matt McCutchen wrote: On Mon, 2009-06-01 at 07:10 +0800, Daniel.Li wrote: On Sun, 2009-05-31 at 14:34 -0400, Matt McCutchen wrote: And one more thing here: If you are going to prepare this batch file, it seems there will be double the workload

Re: how to output verbose information to console when run rsync --daemon?

2009-05-25 Thread Daniel.Li
I used --log-file option, but I just found those connectbla bla... info. No other valuable info like rsync client with -v options. Can anyone help here ? On Wed, 2009-05-06 at 11:42 +0800, Daniel.Li wrote: Dear List, option -v can output information to console. But it seems

Re: weird access function in rsync code.

2009-05-24 Thread Daniel.Li
On Fri, 2009-05-22 at 22:06 +0800, Daniel.Li wrote: On Fri, 2009-05-22 at 13:10 +0200, Paul Slootman wrote: On Fri 22 May 2009, Daniel.Li wrote: On Fri, 2009-05-22 at 10:47 +0200, Paul Slootman wrote: access(/home/admin/test,F_OK) Result: Failed with -1, just mean

Re: Does rsync detect file corruption? -- hard link

2009-05-24 Thread Daniel.Li
On Sun, 2009-05-24 at 16:04 +0200, Mac User FR wrote: Hard-linking an unchanged dir takes very few place. What if a video editor? Lots of work with video files, which is very large, about 500MB per file. Editor only delete or rearrange frames in that file. And then it will be back up 500MB

Re: Does rsync detect file corruption? -- hard link

2009-05-24 Thread Daniel.Li
On Sun, 2009-05-24 at 22:13 -0400, Matt McCutchen wrote: On Mon, 2009-05-25 at 10:09 +0800, Daniel.Li wrote: On Mon, 2009-05-25 at 09:58 +0800, Daniel.Li wrote: What if a video editor? Lots of work with video files, which is very large, about 500MB per file. Editor only delete

weird access function in rsync code.

2009-05-22 Thread Daniel.Li
Dear List, When I use access function to find out if the folder exist in rsync code. I found that it's will NOT act as normal function. I have tried 2 test: a) embedded in rsync function int recv_files(int f_in, char *local_name) and hard coded with following line.

Re: weird access function in rsync code.

2009-05-22 Thread Daniel.Li
at 16:14 +0800, Daniel.Li wrote: Dear List, When I use access function to find out if the folder exist in rsync code. I found that it's will NOT act as normal function. I have tried 2 test: a) embedded in rsync function int recv_files(int f_in, char *local_name) and hard coded

Re: weird access function in rsync code.

2009-05-22 Thread Daniel.Li
On Fri, 2009-05-22 at 10:47 +0200, Paul Slootman wrote: On Fri 22 May 2009, Daniel.Li wrote: When I use access function to find out if the folder exist in rsync code. I found that it's will NOT act as normal function. I have tried 2 test: a) embedded in rsync function int

Re: weird access function in rsync code.

2009-05-22 Thread Daniel.Li
On Fri, 2009-05-22 at 13:10 +0200, Paul Slootman wrote: On Fri 22 May 2009, Daniel.Li wrote: On Fri, 2009-05-22 at 10:47 +0200, Paul Slootman wrote: access(/home/admin/test,F_OK) Result: Failed with -1, just mean the test folder is NOT existed, but it does exist

2.6.9 and 3.0.5 sync error

2009-05-20 Thread Daniel.Li
Dear List, I'm using rsync to sync my data, see below command. And I met this error? I think it might be due to 2.6.9 and 3.0.5 compatible issue. So how to solve the problem? Thanks. rsync -vrtopg --progress --delete --password-file=rsync-password /path/on/pc r...@192.168.0.202::destination

Re: 2.6.9 and 3.0.5 sync error

2009-05-20 Thread Daniel.Li
mai 09 à 13:01, Daniel.Li a écrit : Dear List, I'm using rsync to sync my data, see below command. And I met this error? I think it might be due to 2.6.9 and 3.0.5 compatible issue. So how to solve the problem? Thanks. rsync -vrtopg --progress --delete --password-file=rsync

Delta backup program planned (problem met for adding files)

2009-05-10 Thread Daniel.Li
Dear List, Currently, I'm trying to implement a new feature to rsync. Delta back: Well, it simply just backups diff blocks, and the program is able to restore to any backup points. Current problem met for adding files! But I met a newbie problem here. I'm going to add six files, which are

Re: Delta backup program planned (problem met for adding files)

2009-05-10 Thread Daniel.Li
On Mon, 2009-05-11 at 10:05 +0800, Daniel.Li wrote: When I add an function, compiler prints some error, see below: And I have found that proto.h is generated automatically. So what I have missed here in configuration (adding those six extra files) OK, I have found the problem. If I move

Re: Prepared meta directory for one to many sync

2009-05-06 Thread Daniel.Li
On Wed, 2009-05-06 at 09:20 +0200, Lukas Macura wrote: Hi to all, sorry, but I am not subscribed to this list. I have a question, because I did not find any answer for my problem. If you have so, please let me know! We use rsync to synchronise one-to-many workstations. We use our

how to output verbose information to console when run rsync --daemon?

2009-05-05 Thread Daniel.Li
Dear List, option -v can output information to console. But it seems no effect with rsync --daemon. So, how can I output verbose information to console when running with command rsync --daemon I also tried log file. Seems didn't work. Or is there any other way to read those information, Thanks

[Code study]should we remove if (motd *motd) section?

2009-04-27 Thread Daniel.Li
Dear List, I'm confused when we will run into if (motd *motd)? As I have found that Globals is set 0 during initialization, and I didn't find anywhere else assign the value. If so, should we remove this section? Or there might be some where I missed? Please correct me, if I'm wrong. Thanks

Re: [Code study]should we remove if (motd *motd) section?

2009-04-27 Thread Daniel.Li
On Mon, 2009-04-27 at 12:41 +0200, Paul Slootman wrote: On Mon 27 Apr 2009, Daniel.Li wrote: I'm confused when we will run into if (motd *motd)? As I have found that Globals is set 0 during initialization, and I didn't find anywhere else assign the value. clientserver.c #line

Re: [Code study]should we remove if (motd *motd) section?

2009-04-27 Thread Daniel.Li
On Mon, 2009-04-27 at 19:21 +0800, Daniel.Li wrote: On Mon, 2009-04-27 at 12:41 +0200, Paul Slootman wrote: On Mon 27 Apr 2009, Daniel.Li wrote: I'm confused when we will run into if (motd *motd)? As I have found that Globals is set 0 during initialization, and I didn't find

Re: [Code study]should we remove if (motd *motd) section?

2009-04-27 Thread Daniel.Li
On Mon, 2009-04-27 at 13:34 +0200, Petr Uzel wrote: On Mon, Apr 27, 2009 at 07:27:38PM +0800, Daniel.Li wrote: On Mon, 2009-04-27 at 19:21 +0800, Daniel.Li wrote: On Mon, 2009-04-27 at 12:41 +0200, Paul Slootman wrote: On Mon 27 Apr 2009, Daniel.Li wrote: I'm confused when we

Re: [Code study]append_mode definitions

2009-04-27 Thread Daniel.Li
On Mon, 2009-04-27 at 06:44 -0700, Wayne Davison wrote: On Mon, Apr 27, 2009 at 10:33:02AM +0800, Daniel.Li wrote: Can anyone help to explain definitions of variable append_mode? There are two different type of appending (see the man page) depending on how new rsync is and how much

[Code study]append_mode definitions

2009-04-26 Thread Daniel.Li
Dear List, Can anyone help to explain definitions of variable append_mode? I have found this variable is changed in many places, and quite confused. option.c #line549~550 {append-verify,0, POPT_ARG_VAL,append_mode, 2, 0, 0 }, {no-append,0, POPT_ARG_VAL,append_mode,

Re: Any program flow or software spec for Rsync

2009-04-24 Thread Daniel.Li
On Fri, 2009-04-24 at 07:36 -0700, Wayne Davison wrote: On Tue, Apr 21, 2009 at 02:47:41PM +0800, Daniel.Li wrote: I'm trying to take a look at rsync from code-level. Is there any software spec or program flow that I can get, which might give me a basic understanding about the code

Re: looking for superlifter souce code and related information

2009-04-24 Thread Daniel.Li
On Fri, 2009-04-24 at 07:34 -0700, Wayne Davison wrote: On Tue, Apr 21, 2009 at 10:39:58AM +0800, Daniel.Li wrote: I just googled superlifter and found below link, but I can't get any source code. Sadly, superlifter never achieved lift-off, so there was never any source code

Re: Any program flow or software spec for Rsync

2009-04-23 Thread Daniel.Li
Is there any protocol standard or specs that I can get? Draft is also fine. Thanks. On Tue, 2009-04-21 at 14:47 +0800, Daniel.Li wrote: Dear All, I'm trying to take a look at rsync from code-level. Is there any software spec or program flow that I can get, which might give me a basic

[Code study] lp_motd_file defination? Help

2009-04-22 Thread Daniel.Li
Dear List, Currently, I read this in clientserver.c, line 148. motd = lp_motd_file(); I have googled, but didn't the definition of char *lp_motd_file(void). Can anyone help to explain the following code segment. Many thanks. #line 147 ~ 160, in clientserver.c, version 3.0.5 if (!am_client) {

Re: [Code study] lp_motd_file defination? Help

2009-04-22 Thread Daniel.Li
On Wed, 2009-04-22 at 15:54 +0800, Daniel.Li wrote: Dear List, Currently, I read this in clientserver.c, line 148. motd = lp_motd_file(); I have googled, but didn't the definition of char *lp_motd_file(void). Can anyone help to explain the following code segment. Many thanks. I got

Re: [Code study] lp_motd_file defination? Help

2009-04-22 Thread Daniel.Li
On Wed, 2009-04-22 at 10:14 +0200, Paul Slootman wrote: On Wed 22 Apr 2009, Daniel.Li wrote: Currently, I read this in clientserver.c, line 148. motd = lp_motd_file(); I have googled, but didn't the definition of char *lp_motd_file(void). $ grep lp_motd_file . -R

Re: looking for superlifter souce code and related information

2009-04-21 Thread Daniel.Li
There is a thread palnning for superlifter http://lists.samba.org/archive/rsync/2002-August/003361.html But I don't know if there is any action later? Anyone knows? Or maybe I have take a look at rZync :) On Tue, 2009-04-21 at 10:39 +0800, Daniel.Li wrote: Dear List, I just googled

Any program flow or software spec for Rsync

2009-04-21 Thread Daniel.Li
Dear All, I'm trying to take a look at rsync from code-level. Is there any software spec or program flow that I can get, which might give me a basic understanding about the code? Thanks in advance! -- Daniel -- Please use reply-all for most replies to avoid omitting the mailing list. To

looking for superlifter souce code and related information

2009-04-20 Thread Daniel.Li
Dear List, I just googled superlifter and found below link, but I can't get any source code. http://superlifter.sourceforge.net/ Any help or info on this is mostly appreciated. And sorry for the trouble, if it's NOT allowed to post here. THanks. -- Daniel -- Please use reply-all for most

Re: How many differs in rsync algorithm from Andrew Tridgell's thesis

2009-04-16 Thread Daniel.Li
Is there anyone can help me on this? On Thu, 2009-04-16 at 10:13 +0800, Daniel.Li wrote: Dear List, I would like to take a look at rsync's algorithm. As newbie on this algorithm, I just downloaded Andrew Tridgell's original thesis in 1999. So my question is How many differs in rsync

How many differs in rsync algorithm from Andrew Tridgell's thesis

2009-04-15 Thread Daniel.Li
Dear List, I would like to take a look at rsync's algorithm. As newbie on this algorithm, I just downloaded Andrew Tridgell's original thesis in 1999. So my question is How many differs in rsync algorithm from Andrew Tridgell's thesis. Thanks in advance. -- Daniel -- Please use reply-all

--disable-iconv default configuration

2009-03-16 Thread Daniel.Li
Dear List, I have a quick question about rsync configuration. If I just simply use ./configure, does --iconv supported by server? --disable-iconv disable rsync's --iconv option Thanks. -- Daniel -- Please use reply-all for most replies to avoid omitting the mailing list. To

Re: --disable-iconv default configuration

2009-03-16 Thread Daniel.Li
On Mon, 2009-03-16 at 14:08 +0800, Daniel.Li wrote: Dear List, I have a quick question about rsync configuration. If I just simply use ./configure, does --iconv supported by server? --disable-iconv disable rsync's --iconv option OK, the above problem has been solved. --iconv

Re: --disable-iconv default configuration

2009-03-16 Thread Daniel.Li
On Mon, 2009-03-16 at 14:42 +0800, Daniel.Li wrote: On Mon, 2009-03-16 at 14:08 +0800, Daniel.Li wrote: Dear List, I have a quick question about rsync configuration. If I just simply use ./configure, does --iconv supported by server? --disable-iconv disable rsync's

Re: Rsync Over SSH (Windows XP)

2009-03-14 Thread Daniel.Li
On Sat, 2009-03-14 at 23:41 +1100, David Overton wrote: Hi Henri, 2009/3/14 henri he...@stmargarets.school.nz: In essence I would like to determine if rsync a good choice when it comes to the backup of files on a Windows XP machine? Suggestions such as the following will all be warmly

Re: How to speed up rsync when haveing lots of files

2009-03-04 Thread Daniel.Li
On Tue, 2009-03-03 at 23:46 -0700, lewis butler wrote: On 3-Mar-2009, at 22:55, Daniel.Li wrote: -z is apparently affect the performance when CPU has a lower frequency, like 200MHz or so. When doing rsync, 100% cpu occupied, which limits network performance. You should not use -z

RE: How to speed up rsync when haveing lots of files

2009-03-04 Thread Daniel.Li
On Wed, 2009-03-04 at 13:41 +0100, Boniforti Flavio wrote: This question has been answered. Upgrade to 3.0.5 on both sides. Why has upgrading to 3.0.5 to be done *on both sides*? Any explanation?! As I'm running Debian Lenny (which has 3.0.3), do I have to switch to Debian Sid to get it

Re: How to speed up rsync when haveing lots of files

2009-03-03 Thread Daniel.Li
On Tue, 2009-03-03 at 08:58 -0500, Charles Marcus wrote: On 3/3/2009 8:52 AM, m...@bortal.de wrote: unfortunatelly rsync is beeing REALLY slow and produces a high load when we try to sync lots of files (250 000 small files). What version of rsync are you using? # rsync --version

Re: How to speed up rsync when haveing lots of files

2009-03-03 Thread Daniel.Li
On Tue, 2009-03-03 at 21:26 -0700, lewis butler wrote: On 3-Mar-2009, at 20:33, Daniel.Li wrote: On Tue, 2009-03-03 at 08:58 -0500, Charles Marcus wrote: On 3/3/2009 8:52 AM, m...@bortal.de wrote: unfortunatelly rsync is beeing REALLY slow and produces a high load when we try to sync

Re: rsycing very small files

2009-02-27 Thread Daniel.Li
1. On Fri, 2009-02-27 at 08:20 -0500, Mag Gam wrote: it works. But takes hours to do it. Was wondering if there was a faster way How much speed do u get to backup these files? Average? On Fri, Feb 27, 2009 at 8:04 AM, Paul Slootman paul+rs...@wurtel.net wrote: On Fri 27 Feb 2009, Mag

Time slot back schema

2009-02-23 Thread Daniel.Li
Hi List, Does Rsync support time slot back schema? E.G. (same local folder and remote destination) 1) first backup at 2008-12-24 2) second backup at 2009-02-12 But something bad happens to 2209-02-12, I need to restore files to first backup. Is there any way to do so? Any help is mostly