ray lukas wrote:
Well this link crashes Nutch (redirection problem I would guess but have not
proved it).. I really just need to get my hands on the HTML and I will feed
it into my parsing and indexing systems. For this I just need a crawling
mechanism that will give me the HTML for these types of links. Nutch is,
wonderful but for this overkill and is unable t crawl these links, so I am
looking at Droid as a solution.
I am not archiving anything, I am directly using the html in my java
application. Can Droid crawl this site and return me the correct html. Could
someone try it for me on their droid installation and let me know?
Thanks guys..
http://electricservices.smrated.com/servlet/splocal?m=verizonem&xmid=5060691
&xmcid=-12026&entry_point_id=3079198>
I'm just in the evaluation phase of using Droids myself, but from what
I've found it is quite flexible.
I would try running the SimpleRuntime code with the URL you listed and
see if it gives you back your expected results. If that doesn't work, it
may require some further work.
Richard
