On Saturday 02 August 2003 11:14 pm, Ben Reser wrote: > On Sat, Aug 02, 2003 at 10:07:00PM -0600, Wesley J Landaker wrote: > > All theory and token tests aside, here is some *real* data. I have > > a cooksync-like script (but written in ruby and multithreaded) that > > I've used for at least the last six months, and I have *always* had > > it spit out the stats. In fact, because I'm morbidly curious, I > > *always* look at them. Typically, I get AT LEAST a 20-30% speed, > > often it's more than 50%. > > For reference, here is the stats part of the output for the last > > synching session. For the record, this is typical, not a special > > case or a fluke or something I dragged out of all the bad ones just > > because it looks good. They are *all* this good. > > The problem with this as an example is it includes unversioned > uncompressed files like: > compss > provides > depslist.ordered > > Additionally you're getting the synthesis and hdlist files. > When you take and add those up you come up with about 34MB of data. > When you look at your matching data it comes out to 31MB or so.
Hmm... well, that could be. I've just been believing what rsync tells me. ;) Looking back on my logs, the biggest syncs I've seen in the last few months have been about ~200MB; on most of those I save about ~60-70MB. So I'm saving some bandwidth on some other things somewhere--and, I am seeing some benefit on contrib which doesn't really any meta info files in it, except the synthesis.hdlist... But okay, still, say it turns out that it doesn't work for any of the RPMs, and I'm only saving badwidth on the deplist and friends--I'm still *overall* getting the speedups that I mentioned for the kind of synchronization schedule that I'm using. =) If I just ftp'd everything, that would be ~30% data I downloaded every time, whether or not it's RPM data. I guess if what you say is true, I'd get less of an [apparent?] benefit if I didn't sync as often, since the RPM data to depslist-type data ratio would be higher. > Makes me wonder if the hdlist and synthesis files aren't rsyncing > well. I should run some experiments to see how well that works... > Unfortunately the explanation of the format in the packdrack man page > is rather lacking. I've saved a copy of my base dir and I'll see > what I can come up with for testing tomorrow. > > But I'm highly suspicious that all of that speed up is from moving > the files around. It just doesn't fit the data and the protocol... You could be right; nevertheless, I'm seeing a speed up from somewhere. Perhaps I'll try using vanilla rsync for a week with stats reported and see if it looks like it makes a large difference vs. the cooksync method. If we kind find out exactly where it's coming from, maybe we'll see that extra RPM moving voodoo isn't as beneficial as some of us have been imagining. -- Wesley J. Landaker - [EMAIL PROTECTED] OpenPGP FP: 4135 2A3B 4726 ACC5 9094 0097 F0A9 8A4C 4CD6 E3D2
