> I am working on a robot develpoment, in java,. > We are developing a search engine....almost the > complete engine is developed... > We used java for the devlopment...but the performance > of java api in fetching the web pages is too low, > basically we developed out own URL Connection , as > we dont have some features like timeout... > supported by the java.net.URLConnection api ..
Look at Java 1.4, it addresses these issues (socket timeouts, non-blocking IO, etc.) > Though there are better spiders in java..like > Mercator..we could not achive a better performance > with our product... I thought Mercator numbers were pretty good, no? > Now as the performance is low..we wanted to redevelop > our spider..in a language like c or perl...and use > it with our existing product.. You could look at Python, Ultraseek was/is written in it, from what I remember. Also, obviously Perl has been used for writing big crawlers, so you can use that, too. > I will be thankful if any one can help me choosing > the better language..where i can get better performance.. Of course, the choice of a language is not a performance panacea. Otis __________________________________________________ Do You Yahoo!? Yahoo! Sports - live college hoops coverage http://sports.yahoo.com/ -- This message was sent by the Internet robots and spiders discussion list ([EMAIL PROTECTED]). For list server commands, send "help" in the body of a message to "[EMAIL PROTECTED]".
