Henri Sivonen wrote:
That seems like a bad optimization. Adding an off-the-shelf HTML parser to a bot is much easier than tuning the general crawling functionality and task-specific functionality of a bot.
I'll be convinced when I see support for this being integrated into the MacOsX and Microsoft Windows WebDAV clients.
BR, Julian
