hi all, I index several domains that are not under my control, and I have a list of which domains were successfully indexed so the maintainers of those domains can check to see if they are in the database.
The problem is, even though I use the -s flag to htdig to get a summary of servers visited and pages found on each, the output that follows pushed the summary far past my scrollback buffer's limit, so I couldn't read it. Is there a way, other than redirecting all output to a file and reading it afterwards, to get that summary? I run it with one -v in a terminal, and glance at it from time to time. (I still tweak the configuration regularly, so I want to keep an eye on it to catch infinite dynamic directory trees and duplicate crawling eg www.something.tld and something.tld to add those to my config file to prevent it the next time around. And the domains I crawl aren't under my control, so they may change something to create a new infinite dynamic directory tree without telling me.) db.urls appears to be a list of all urls it sees, whether or not they were within limits or the server was responding, so that isn't terribly useful to me. I looked through the FAQ, but 'list' and 'url' occur so many times I couldn't find anything related to what I want, and I couldn't use the search engine on your main page because "The requested URL /cgi-bin/htsearch was not found on this server." Thanks in advance, -Rhonda -- www.write-on.indy || write-on.homeip.net \/ http://history.ubcengineers.ca/ Discuss the art and craft of writing /\ UBC Engineers History Project That's the problem with world domination. Nobody is willing to wait for it anymore, work slowly towards it, drink more and enjoy the ride more. _______________________________________________ htdig-general mailing list <[EMAIL PROTECTED]> To unsubscribe, send a message to <[EMAIL PROTECTED]> with a subject of unsubscribe FAQ: http://htdig.sourceforge.net/FAQ.html
