It would seem that one could use to run many hours of
spidering in a single hour by having a bunch of xen virtual machine
instances set to do this. If one started up and shut down instances for the
crawl, it would not seem to be expensive to do a lot of work concurrently on
a big crawl. Is this practical? Why or why not?

Having a virtual server farm that apears just when I need it and only costs
me 10 cents / machine per hour and zero when I'm done with my spidering
sounds like something I should explore apart from just the coolness of it.

What about actually running the web front end search site on EC2? Would that
be wise? Will I get the performance that I need to be seen as a responsive

When they talk about Amazon S3 (Simple Storage Service), is this raw disk
space that they are selling or backed up, guaranteed to be there storage?

Does anyone on the list of actual experience with either of these offerings
or any competitive offerings from anyone else?

I assume that Amazon has a huge infrastructure with great connectivity. Do I
get the benefit of all that if I use EC2 really, in practical reality? Will
it actually help me keep my search index up to date?


Reply via email to