I found some time in the past week to work on a simple test app that would hopefully help to answer a few questions that came up recently:
1. Can a single-process generator+receiver work well? (Looks good so far, but I haven't run any multi-processor timing tests yet.) 2. How easy is it to use librsync? (Pretty easy.) 3. How small would a transfer-only tool be? (It's currently around 1400 lines of C code, not counting the librsync code. It was around 900 lines when I first considered releasing a simple working version, but it keeps growing as I flesh out the more advanced features.) 4. Should rsync be separated into a scanning tool and a transfer tool? Or should it contain both bits but also allow the user to override the scanner to fully control what gets transferred? Or should we just try to optimize the current protocol? (No answers yet, but I'm leaving toward the 2nd option above.) My test tool takes in commands on stdin and outputs messages on stdout. It forks a second process as specified via the commandline (which can be any command that runs another rsync_xfer, either locally or remotely). It then allows you to send AND/OR receive any files you specify (as well as delete files, mkdir directories, etc.). Keep in mind that this tool does not attempt to do any of the "scan both systems, looking for files that differ" task. The code is still fairly young and while some of it is pretty good, other bits show signs of being written in haste. I've tested it on a small number of scenarios so far, but nothing exhaustive. Commands accepted by the tool on stdin (* means not yet tested): cd REMOTE_DIR [LOCAL_DIR] chdir both sides at once tmpdir REMOTE_PATH [LOCAL_PATH] where temp-files go get REMOTE_FILE [LOCAL_FILE [BASIS_FILE]] rsync to the local system put LOCAL_FILE [REMOTE_FILE [BASIS_FILE]] rsync to the remote system mvget REMOTE_FILE [LOCAL_FILE [BASIS_FILE]] get, then delete REMOTE_FILE mvput LOCAL_FILE [REMOTE_FILE [BASIS_FILE]] put, then delete LOCAL_FILE del FILE delete a remote file ldel FILE delete a local file md DIR create a remote directory lmd DIR create a local directory ln OLDNAME NEWNAME create a remote hard link lln OLDNAME NEWNAME create a local hard link sln OLDNAME NEWNAME create a remote symlink lsln OLDNAME NEWNAME create a local symlink mkdev NAME NUMBER create a remote device* lmkdev NAME NUMBER create a local device* quit quit Spaces in filenames need to be backslash-quoted, as do backslash characters. E.g. get This\ File.txt That\ File.txt You run the program like this: rsync_xfer -vv ssh remote.com rsync_xfer -s This starts up a local rsync_xfer process in double-verbose mode, and tells it to run the "ssh remote.com rsync_xfer -s" command. You can make this latter command anything you like, as long as it starts up an rsync_xfer with the slave (-s) option. If you're feeling brave and you'd like to try it out, feel free, but treat it like the pre-alpha code that it is. Also keep in mind that every time you tell the program to switch from get to put, all the current outstanding get/put jobs must run to completion before any new jobs start (which can slow down the transfer by reducing the pipe-lining of data). Some of the things yet to do: - Need a way to override the per-file attributes on files we send (it currently preserves the attributes on each source file). - Need a way to specify/set attributes for non-transferred files, new directories, and devices. - It needs a way to output transfer statistics. - There needs to be a timeout option. - It needs some of the error-checking to be polished up. - Some fatal errors might be better as warnings (if done right). - There's no retry check if the file changes during the send. - We need to catch SIGPIPE. - There needs to be better handling of partially-transferred files. - The code needs to be broken up into multiple files. - There's no configuration support (it currently compiles on a modern Linux system). - We might want an option that tells us to connect via socket to a particular hostname (instead of running a command). - The code could use some more verbose-output messages. - It needs more comments. The code is here: http://www.clari.net/~wayne/rsync_xfer.c You need to have librsync installed or available. You compile the code as you would expect: gcc -g -Wall -o rsync_xfer rsync_xfer.c -lrsync ..wayne.. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html