Hey Joshua,
On Aug 15, 2006, at 1:33 PM, Joshua Schachter wrote:
> All kinds of subtlety here. For example, what to do if the site
> happens
> to be down while we check it? What about respecting robots.txt etc?
You could explicitly only use HTTP return codes. That is to say, only
take action for sites that are accessible, and return certain codes,
eg: 301 (permanently moved), 404 (not found), and 410 (gone). A site
outage would (hopefully) be returning 500 or nothing at all. Sites
that weren't accessible (DNS, no route, etc) would be left alone.
I'm not sure what to do about robots.txt, which also raises a more
fundamental question of whether d.i.u really should be actively doing
anything at all, as opposed to passively accepting submissions.
Perhaps people who want additional features like this should just
work towards client-side applications that use the d.i.u API, and
leave the site itself as a repository.
- siege
---
Christopher (siege) O'Brien
[EMAIL PROTECTED]
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/ydn-delicious/
<*> To unsubscribe from this group, send an email to:
[EMAIL PROTECTED]
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
