Oh, syncing from a pipe on one end to a file on the other end is a different story. That I can see a use for. However, it's not really what the rsync command is designed for; it's designed for handling lots of files. I would think a simple tool based on librsync would be better suited. If we did want rsync to support it, I wouldn't lump it with reading of other device files. A syntax of specifying "-" as the source file would make some sense, but a whole lot of options (especially most of the ones inferred by "-a") won't make any sense in that situation, and the destination has to be a file and cannot be a directory. I think it would probably have too great of an impact on the rsync code to be worth supporting.
Syncing an entire raw partition is not a good idea unless you plan to have it unmounted during the entire operation, because otherwise you're likely to get some corruption in the copy. - Dave On Fri, Aug 30, 2002 at 02:48:20PM +0300, Eran Tromer wrote: > Hi, > > Thanks for your explanation. Before answering, I'll note that (as > mentioned in a followup) my patch has some unintended debugging cruft > and I'll gladly provide a clean patch if anyone's interested. > > Well, here's my personal motivation. I have a remotely located server > with a 6GB disk, and also a newer machine a 80GB disk. I wish to keep a > full backup of the server filesystem on the other machine. A fairly > common setting, I imagine. Originally I rsynced the servers root > directory into a directory on the backup machine. This has two main > disadvantages: > > 1. To get permissions and special files right, I had to run the receiver > side as root. From a security viewpoint, this is already nasty in the > extreme, but it's much worse than that, because now the backup machine > contained all sort of spooky device files and suid root binaries and > whatnot, created by rsync. If the server was compromised, in all > likelyhood the backup machine would go down next. Trusting permissions > on the parent directory of the rsync destination directory is not quite > enough for such stuff -- these things *have* been bypassed before. > > 2. The target machine has a different filesystem. In my case it was only > a larger block size, which caused the backup to consume *much* more > diskspace than needed. In other cases, it could mean losing ACLs or > resource forks or whatever. > > Now the beautiful thing is that tar handles all those things. All I > really want to do is to have an up-to-date "server.tar" file on the > backup machine, containing the output of running "tar cf - /" on the > server. But to do this with unpatched rsync, I needed to first create > the tar file on the server, which would *double* its diskspace requirements. > > Nowadays, I pipe the output of "tar cf - /" into a pipe that's given as > as a filename to a patched sender rsync; the patched receiver rsync then > updates the corresponding huge file on the client. I have a rotten ADSL > connection with terrible upstream speed so I increased the block size to > 10K, but the daily updates of these 6GB still take about 10 minutes (the > bottleneck is the server disk I/O time). Pretty fast, for something > completely stateless that's perfectly immune to wrong timestamps and > suchlike. > > The SQL thing I mentioned is related, but here there isn't even a "just > rsync the files directly" alternative. Again, say you have a server > running some database server, and you want to take snapshots for backup > purposes, or whatever. Rsyncing the actual database files, if at all > possible for your RDBMS, will give you a corrupted database. However, if > you have a utility to dump the database into a flat file (e.g., > mysqldump of MySQL) then you can use that to get a consistent snapshot, > and rsync that snapshot. As before, to prevent the need for a huge > temporary file it's really nice to be able to pipe the output of the > dump utility directly into rsync. > > Someone else had a different scenario, involving backing up a complete > disk partition at the block level. This also sounds quite useful, and > again quite impossible with unpatched rsync without a lot of extra > diskspace outside that partition. > > As you can see, all of these backup-related scenarios make sense even > without the ability to update special files (which is obviously > problematic, though as I suggested a while ago, perhaps many common > cases *can* be efficiently handled pretty easily, especially data is > frequently updated but seldom shifted --- think /dev/hda). > > Last but not least: this is the Unix way. It provides much power and > flexibility to the user, hence it is a Good Thing. I find my local > patched rsync very useful, and would experience eternal merriment if > everybody got that feature. > > "device files" come into the picture simply because a pipe into stdin > (or any other FD) is seen a special file (on Linux, at least), and the > "--devices" command to rsync affects all special files, so > "--read-devices" appears appropriate. "--read-special" would be more > accurate, but less consistent. > > BTW, I noticed that statistics reporting isn't working very well when > special files are read using my --read-devices patch, despite my efforts > to get some reasonable behavior (the problem is that the file size isn't > known until the whole file was read and sent). Probably fixable, and not > a major issue when someone needs the feature, methinks. > > Regards, > Eran > > > > > Dave Dykstra wrote: > >Similar patches have been submitted before but they've always been > >rejected. I'm sorry that you spent so much time on this one, although > >perhaps it's been useful to you so far. > > > >As you know, rsync won't be able to write to such devices because it needs > >to work on a temporary copy; that limits the usefulness of reading from > >them greatly. I'm not at all convinced that it's worths supporting it in > >rsync. > > > >I can't read your examples in the rsync.fom right now because it's broken, > >so I don't understand your motivating examples. How can you rsync into a > >tar file, and what do SQL databases have to do with device files? > > > >- Dave Dykstra > > -- > To unsubscribe or change options: > http://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html