> > It may be even nicer to use some DB implemented in Java, such as > > HyperSQL (I think that's the name) or Smyle > > (https://sourceforge.net/projects/smyle/) or Berkeley DB > > (http://www.sleepycat.com/), although MySQL may be simpler if you > want > > to create a crawler that can be run on a cluster of machines that > share > > a central link repository. > > Hm, I'll think about it. But MySQL seems to be the KISS way... > I don't think a central link repository makes sense. Looks like a > bottleneck to me.
Well, yes, it could become a bottleneck. However, your crawler is not distributed (yet?), so we don't have to waste time talking about hypothetical situations. Otis __________________________________________________ Do You Yahoo!? Yahoo! - Official partner of 2002 FIFA World Cup http://fifaworldcup.yahoo.com -- To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]> For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>
