On Fri, 22 Feb 2002, Gilles Detillieux wrote:
+ > On a slight change of tack, perhaps better on the list? ... Did my request
+ > for a facility to insert a timestamp into the -v output stream ever get
+ > onto the to-do list? I ask because it would have been useful to tell when
+ > a server dropped out last night. I suspect htdig managed to catch it
+ > exactly as it did a 4am logrotate and so my database lost 3,900 documents.
+
+ Well, I don't recall the request, so it never got on my own to-do list,
It may been tacked-on to another enquiry. I don't recall my pursuing it
very actively.
+ and the to-do list on the web site isn't that current. If you run with
+ -vvv, you should get output showing the Date headers put out by the HTTP
+ server, but I suppose that's not quite ideal.
No! Not ideal, with 200,000 documents I have problems going above -vv!
+ How frequently should these timestamps be put out? Every document or
+ every few minutes?
I suppose a time interval would be the best solution but not being a C++
man myself, I was thinking that if I ever found time to venture into the
code myself I would look for the code where -v generates the output
starting <n1>:<n2>:<n3><url> and add a timestamp just ahead of that. Every
record would be rather too much!! I was thinking of maybe every 500 or
1000 pages with a simple mod/remainder check on <n2>.
My digging seems to index at about 10k pages/hour and 10-20 stamps per
hour would keep me happy! My objective is not to pinpoint time exactly but
just to give webmasters a fighting chance of spotting a log entry when I
ask if their server croaked over night.
regards,
Malcolm.
[EMAIL PROTECTED] http://users.ox.ac.uk/~malcolm/
_______________________________________________
htdig-general mailing list <[EMAIL PROTECTED]>
To unsubscribe, send a message to <[EMAIL PROTECTED]> with a
subject of unsubscribe
FAQ: http://htdig.sourceforge.net/FAQ.html
