Brian
You raise several issues here, and I'm going to try to address them one
by one.
(Aug 02 2005 10:33) Brian Wilson wrote:
> >That's a bold statement. And bold statements should not go
> >unanswered.
>
> HA! That's NOTHING! Try this: "grep is 5000 times faster than swup".
The statement related to a situation where using swup in combination
with grep. Swup is not a general search tool, but a networked
application limited by network bandwidth. My point was that in that
scenario, you never win by using grep. ;)
> On my SLOWEST 180 Mhz computer, swup = 5-20 seconds, grep = .01
> seconds. On my fast computer, grep <= .004 seconds. Swup? 5-20 seconds.
>
> What your test showed was that swup is so SLOW even adding grep did
> not make it WORSE. :-)
That's a point. Swup cannot be faster than your internet access time,
unless everything it cached. At that point it's never faster than your
disk access time.
It cannot be done another way except the 'apt' way of separating the
download part from other operations. The problem with doing that is that
it's more work for the user to get consistant responses.
However, after the first search, everything is cached, so any subsequent
searches will be 'freebies' only downloading the .md5 files to verify
that the local copy is up to date.
This is of course slower than grep, but as my timing on my private
hardware showed, it's only a matter of 5 seconds. I have that kind of
time.
> You tested "swup --search" vs "swup | grep", but what we're doing is
> "swup > file" (slow part) followed by multiple grep's on the same file
> (fast). If I am sure I know the package I want, I do "swup --install"
> right away. If I am 98% sure I use "swup --search-package".
Well, you never stated that in your original mail, so I had to base my
test on the example in the other postings in that same thread. :)
> But that's not the real point; I really don't think it's bad on clock
> time; the point is PERCEIVED time.
>
> SOMETIMES swup downloads files (you lose) and sometimes not (you win);
> on top of that, every run involves a random time element as it goes
> over the Internet so its runtime is always inconsistent but always
> slower than grepping a file. I KNOW grep is ALWAYS consistently
> fast. On my computers grepping a local file is always AT LEAST 500
> times faster than running swup. YMMV -- if your drive is slower than
> your Internet connection.
A agree. When doing it the way you now describe, grepping is that much
faster (untested).
> What does it take to mirror the RDF's to my LAN and turn off swup's
> caching thing entirely? I would guess if I could eliminate "the
> checking for updates" phase in swup entirely and just always use a
> copy somewhere on my LAN instead, swup would be consistent and fast.
To set up a sync of the official mirrors you use rsync:
***********************
#!/bin/sh
DEST='/home/ftp/pub'
RSYNC='rsync -q -avH --delete'
HOST='rsync.trustix.org'
TARGETS='trustix tsldev'
pushd $DEST > /dev/null
for TARGET in $TARGETS; do
$RSYNC $HOST::$TARGET $TARGET/
done
popd > /dev/null
***********************
The 'turn off cache part' is not possible today. And even so, you would
not gain anything from it, as that means that information had to be
polled from your local mirror every single time. This means downloading
and temporary storing on disk.
So turning off cache would only be interesting if:
1. You have better access time over the network than on your disks.
2. You put /var/spool/swup and /var/spool/cache on tempfs in RAM
Actually, putting the cached files in RAM will speed things up a bit on
slow IO systems. :)
> I am guessing that you at TSL swup more often than most people so you
> developers don't see the "downloading RDF" message as often as we
> users do.
I only swup in cron and when someone forces me to test stuff, like
speed. ;)
> Your perception is that it is already consistent and fast. Maybe I
> should just be a developer.
I don't think your conclusion is correct. However, if 5 seconds is too
much for you, then I would really consider setting up a local mirror,
and maybe even commenting out the official mirrors. (Not recommended as
you would only rely on the local sync job for updating systems.)
The ram trick would only speed things up if you have extremely good LAN
bandwidth. If you have fast IO access and slow LAN access, I would
consider setting up the mirror on the server itself, and using
file:///home/ftp/pub/trustix/.... as location for the local site.
> Aside: I finally deleted gatech.edu from my swup.conf files. Once too
> often it said "there are too many connections from your location." and
> died in the middle of swup -install. Deleting that entry has sped
> things up a bit; now search-packages never takes 3-5 minutes as it
> used to sometimes. :-)
If you had that kind of access time I understand you would use the grep
trick. However, that kind of access time is not what I consider normal.
> ibiblio still seems slower than trustix.org was but it never dies 1/2
> way like gatech. (Hmmm... what's the point of me using the mirrors if
> they are always slower than the Trustix site?? I might just go back
> to the source.)
Good point. :)
c
--
Christian H. Toldnes
Trustix Developer
_______________________________________________
tsl-discuss mailing list
[email protected]
http://lists.trustix.org/mailman/listinfo/tsl-discuss
