At 08:21 PM 17/01/2002 -0800, Gary Funck wrote:
>Can someone recommend a simple, efficient way to monitor a list of various
>web pages (or in some cases, a hierarchy of pages) for changes, and then
>send an alert mentioning which pages have changed?
A hierarchy of pages can really only be treated as an extended list of
individual pages. Unless the content is fairly contrived, changing a
low-level page doesn't impact other pages.
Overall though: Depends on how detailed you want to get.
HTTP (usually) provides a Last-Modified header, you can just monitor that.
Doesn't work for bad servers, and most dynamic pages (since they only
spring into existance at your request).
Next step is to download pages, say just into memory and calculate a
signature (MD5 hash, say) and store that and compare to previous versions.
Doesn't take a lot of storage if you want to monitor a lot of pages.
Top level is to maintain a copy, say in a timestamped directory, and do a
local comparison. If you want to generate 'diff()s' then this is probably
the only way to go. You only need to store versions as they change, so a
download-compare-keep/toss cycle would suffice, and minimise what you need
to store. It then depends on how many versions back you want to view.
There's probably other ways to do it too, these were the first that leapt
to mind...
Cheers,
Markus
Markus Buchhorn, Information Infrastructure Services, | Ph: +61 2 61258810
[EMAIL PROTECTED], mail: CompSci,CSIT Bldg #108|Fax: +61 2 61259805
Australian National University, Canberra 0200, Australia|Mobile: 0417 281429
